-- Logs begin at Fri 2022-09-16 07:15:51 UTC, end at Fri 2022-09-16 08:24:57 UTC. -- Sep 16 07:45:05.989887 dv-cinder-740146-20-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 16 07:45:06.000922 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 16 07:45:06.003122 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 16 07:45:06 2022] *** Sep 16 07:45:06.003122 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 16 07:45:06.003122 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 16 07:45:06.003122 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: nodename: dv-cinder-740146-20-iscsi Sep 16 07:45:06.003122 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: machine: x86_64 Sep 16 07:45:06.003122 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: clock source: unix Sep 16 07:45:06.003122 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: pcre jit disabled Sep 16 07:45:06.003122 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: detected number of CPU cores: 4 Sep 16 07:45:06.003122 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: current working directory: / Sep 16 07:45:06.003122 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: detected binary path: /usr/bin/uwsgi-core Sep 16 07:45:06.003122 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: your processes number limit is 47680 Sep 16 07:45:06.003785 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: your memory page size is 4096 bytes Sep 16 07:45:06.003785 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: detected max file descriptor number: 2048 Sep 16 07:45:06.003785 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: lock engine: pthread robust mutexes Sep 16 07:45:06.003785 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: thunder lock: enabled Sep 16 07:45:06.003785 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 16 07:45:06.003785 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 16 07:45:06.042162 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: Python main interpreter initialized at 0x55e32ba3a9c0 Sep 16 07:45:06.042162 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: python threads support enabled Sep 16 07:45:06.042162 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: your server socket listen backlog is limited to 100 connections Sep 16 07:45:06.042162 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: your mercy for graceful operations on workers is 90 seconds Sep 16 07:45:06.042483 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: mapped 403077 bytes (393 KB) for 2 cores Sep 16 07:45:06.042558 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: *** Operational MODE: preforking *** Sep 16 07:45:06.042861 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: *** uWSGI is running in multiple interpreter mode *** Sep 16 07:45:06.042861 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: spawned uWSGI master process (pid: 81563) Sep 16 07:45:06.043055 dv-cinder-740146-20-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 16 07:45:06.043875 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: spawned uWSGI worker 1 (pid: 81573, cores: 1) Sep 16 07:45:06.043976 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: spawned uWSGI worker 2 (pid: 81574, cores: 1) Sep 16 07:45:06.046706 dv-cinder-740146-20-iscsi devstack@keystone.service[81563]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 16 07:45:10.415805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 07:45:10.415805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: from cryptography.utils import int_from_bytes Sep 16 07:45:10.416604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 07:45:10.416604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: from cryptography.utils import int_from_bytes Sep 16 07:45:10.417036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 07:45:10.417036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: from cryptography.utils import int_from_bytes Sep 16 07:45:10.417166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 07:45:10.417166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: from cryptography.utils import int_from_bytes Sep 16 07:45:10.466517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 16 07:45:10.466733 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 16 07:45:10.466828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 16 07:45:10.466919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 16 07:45:10.466991 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] command line args: [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 16 07:45:10.467114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] command line args: [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 16 07:45:10.467190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 16 07:45:10.467367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ================================================================================ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 16 07:45:10.467449 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 16 07:45:10.467577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ================================================================================ {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 16 07:45:10.467653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] admin_token = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.467833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] admin_token = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.467911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config_dir = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.468037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.468112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] config_dir = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.468220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config_source = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.468302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.468401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] debug = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.468477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] config_source = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.468593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] debug = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.468715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.468928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.469022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.469086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.469151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.469215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.469353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.469441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.469605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.469680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.469837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.469911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.470020 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_config_append = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.470099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_config_append = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.470196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.470270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.470440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_dir = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.470516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_dir = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.470582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_file = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.470651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.470937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.471036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.471107 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.471167 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.471311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.471403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.471522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.471603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.471693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.471818 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.471909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.471989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.472101 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.472201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.472346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.472443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.472577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.472683 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.472810 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.472894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.473038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.473162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.473277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.473386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.473496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.473605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.473720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.473842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.473987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.474104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.474469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.474584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.474745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] publish_errors = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.474859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] publish_errors = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.474975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.475097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.475228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.475347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.475458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.475567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.475675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.475854 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.475968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.476085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.476187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] standard_threads = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.476295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] standard_threads = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.476417 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.476535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.476647 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.476769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.476882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.476999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.476999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] use_journal = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.477221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_journal = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.477339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] use_json = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.477456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_json = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.477567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] use_stderr = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.477692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_stderr = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.477811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] use_syslog = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.477922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_syslog = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.478032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.478141 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 07:45:10.478257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.478375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.478496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.478613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.478749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.478869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.478987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.479174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.479289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.479399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.479516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.479627 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.479743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.479865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.479983 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.480100 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.480260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.480374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.480491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.480611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.480724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.480834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.480949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.481067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.481178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.481299 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.481412 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.481528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.481647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.481765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.481876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.481995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.482592 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.482731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.482855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.482974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.483090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.483212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.483325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.483441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.483558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.483677 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.483794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.483913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.484026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.484142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.484259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.484376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.484488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.484606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.484720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.484836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.484948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.485057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.485242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.485357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.485467 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.485577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.485687 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.485806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.485929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.486042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.486158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.486270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.486379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.486489 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.486610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.486781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.486903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.487015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.487126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.487236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.487353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.487466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.487576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.487691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.487787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.487888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.487989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.488083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.488175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.488272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.488377 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.488471 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.488571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.488665 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.488777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.488876 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.488951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.489094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.489219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.489332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.489444 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.489554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.489664 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.489767 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.489887 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.490001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.490109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.490219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.490336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.490448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.490570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.490704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.490821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.490943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.491058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.491170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.491287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.491407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.491527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.491630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.491740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.491856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.491968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.492091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.492205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.492315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.492432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.492544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.492661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.492784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.492898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.493009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.493126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.493245 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.493366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.493484 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.493605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.493724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.493837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.493948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.494064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.494177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.494299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.494413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.494524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.494635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.494786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.494910 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.495025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.495137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.495249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.495360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.495471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.495592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.495712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.495825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.495926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.496035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.496145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.496331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.496448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.496559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.496671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.496781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.496892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.497011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.497125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.497236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.497348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.497459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.497570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.497689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.498435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.498564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.498692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.498808 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.498919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.499041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.499155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.499275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.499390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.499502 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.499613 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.499724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.499842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.499962 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.500076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.500187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.500299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.500410 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.500522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.500641 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.500755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.500867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.500979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.501090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.501201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.501320 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.501442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.501556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.501668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.501778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.501931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.502054 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.502168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.502280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.502399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.502513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.502626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.502775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.502891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.503002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.503113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.503229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.503346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.503467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.503582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.503693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.503805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.503916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.504036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.504175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.504340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.external = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.504466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.external = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.504581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.504695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.504807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.504920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.505042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.505157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.505269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.password = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.505408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.password = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.505522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.token = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.505643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.token = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.505766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.505880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.506005 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.506176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.506293 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.506435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.506560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.506688 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.506805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.506918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.507031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.507154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.507277 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.507391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.507543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.507661 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.507776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.507888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.508008 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.508129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.508254 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.508369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.508488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.508601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.508721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.508842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.508957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.509068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.509181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.509292 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.509411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.509411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.509631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.509631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.509885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.510002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.510123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.510123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.510335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.510335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.510569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.510569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.510912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.510912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.511129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.511129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.511340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.511452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.511576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.511691 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.511803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.511915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.512034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.512147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.512267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.512381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.512494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.512606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.512718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.512830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.512951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.513067 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.513183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.513296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.513409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.513521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.513644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.513759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.513872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.514012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.514128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.514240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.514360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.514480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.514594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.514719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.514835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.514959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.515075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.515189 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.515301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.515414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.515526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.515646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.515760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.515879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.515993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.516105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.516218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.516338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.516452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.516564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.516676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.516789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.516901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.516901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.517122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.517122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.517333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.517445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.517626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.517747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.517862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.517975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.518087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.518087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.518298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.518298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.518519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.518631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.518765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.518879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.518990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.519114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.519229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.519343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.519455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.519567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.519685 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.519807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.519921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.520033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.520145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.520258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.520371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.520491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.520605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.520724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.520838 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.520950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.521061 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.521184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.521298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.521412 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.521524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.521642 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.521756 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.521876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.521990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.522103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.522215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.522327 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.522438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.522562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.522728 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.522845 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.522969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.523084 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.523201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.523321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.523435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.523548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.523660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.523771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.523883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.523883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.524103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.524103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.524312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.524425 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.524536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.524658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.524772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.524884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.524996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.525108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.525219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.525339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.525454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.525566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.525679 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.525790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.525909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.526056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.526169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.526281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.526393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.526505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.526625 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.526754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.526868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.526981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.527093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.527205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.527328 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.527444 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.527556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.527667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.527786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.527900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.528021 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.528136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.528248 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.528360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.528472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.528583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.528703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.528817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.528929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.529041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.529153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.529265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.529384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.529498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.529611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.529729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.529844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.529957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.530077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.530191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.530303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.530428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.530585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.530717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.530845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.530962 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.531074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.531186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.531297 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.531408 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.531527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.531641 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.531753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.531864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.531976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.532088 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.532207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.532321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.532433 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.532545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.532657 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.532775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.532964 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.533082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.533198 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.533310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.533422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.533533 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.533653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.533767 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.533879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.534006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.534006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.534263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.534263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.534484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.534597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.534737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.534852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.534964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.535089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.535205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.535317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.535429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.535540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.535652 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.535774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.535902 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.535902 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.536148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.536259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.536371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.536491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.536606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.536718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] role.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.536829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.536941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] role.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.537052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.537173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.537287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.537400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.537511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.537623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.537735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.537860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.537975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.538087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.538200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.538313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.538432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.538547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.538659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.538792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.538915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.539030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.539149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.539265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.539377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.539490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.539602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.539715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.539835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.539950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.540062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.540175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.540288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.540288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.540513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.540633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.540747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.540747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.540971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.541085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.541204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.541319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.541431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.541544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.541661 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.541774 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.541895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.542009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.542121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.542233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.542345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.542458 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.542577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.542746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.542862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.542975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.543086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.543203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.543327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.543443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.543556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.543668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.543779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.543898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.544012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.544125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.544236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.544348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.544459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.545076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.545193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.545306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.545417 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.545531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.545642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.545762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.545877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.545989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.546107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.546221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.546333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.546453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.546568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.546696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.546811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.546924 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.547036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.547159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.547274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.547386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.547498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.547610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.547721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.547841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.547955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.548067 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.548179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.548291 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.548402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.548522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.548636 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.548750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.548869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.548991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.549104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.549216 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.549328 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.549452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.connection = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.549577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.549692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.549805 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.549917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.550029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.550140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.550263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.550378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.550490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.550602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.550726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.550843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.550966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.551082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.551194 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.551306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.551418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.551530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.551716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.551834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.551947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.552059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.552171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.552171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.552390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.552504 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.552617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.552728 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.552840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.552952 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.553072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.553190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.553302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.553414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.553526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.553637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.553757 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.553871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.553984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 16 07:45:10.554098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 07:45:10.554098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 16 07:45:10.581952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.582182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.582342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.582426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.582577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.582832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.590690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.591159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.618907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.619134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.619385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.619620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.673347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.673496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.723303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.723496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.723630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.723720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.887192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.887383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.887528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.887636 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.887718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.887794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.887868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.887942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.888024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.888143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.888228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.888384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.920690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.920918 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.921041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.921127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.921276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.921359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.921505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.921589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.921724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.921807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.921938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.922020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.922149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.922359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.933274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.933374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.946089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.946320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.946409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.946539 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.946623 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.946781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.946884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.946996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.947079 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.947206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.947289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.947441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.947533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.947654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 07:45:10.964456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.964638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.964723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 07:45:10.968617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 07:45:10.968719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.968804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.968882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 07:45:10.968958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 07:45:10.969032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.969120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.969199 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.969273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.969347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.969420 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.969502 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.969578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.976340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.976572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.976657 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.976851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.976949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.977126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.977210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.977410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.984704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:10.984978 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:10.987616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:10.987879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:10.987962 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.988171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.988265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.988453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.988535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.988741 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.988831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.989047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.989148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.989314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.989395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.989586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.989667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.989854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.989935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.990126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.990206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.990392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.990480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:10.990659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.014339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.014439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.015826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.015917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.016112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.016197 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.020765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.020850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.021052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.021150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.021333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.021416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.027692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.027786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.027980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.028064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.033580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.033667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.033879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.033961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.034184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.034268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.034459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.034541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.034760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.034855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.035041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.035124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.035311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.035394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.035581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.035664 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.051524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.051617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.051803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.051886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.055738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.055934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.058988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.059185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.059268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.059483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.064069 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.064255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.067555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.067699 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.070787 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.070924 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.071086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.071225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.071376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.071511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.071658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.071793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.071949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.072070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.072239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.072358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.088612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.088708 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.088902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.088986 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.089167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.089249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.089431 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.089514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.096786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.096880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.098314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.098397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.099858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.099955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.100147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.100229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.102826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.102916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.103112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.103208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.103394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.103477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.103678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.103759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.104067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.104150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.115207 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.115301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.115492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.115588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.115777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.115860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.116061 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.116144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.116438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.116524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.116719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.116801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.116997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.117083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.132357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.132450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.135271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.135359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.138126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.138229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.140175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.140259 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.141630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.141713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.141912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.141995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.142304 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.142385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.148656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.148740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.150104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.150195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.153002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.153086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.153402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.153501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.153691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.153775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.154497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.154594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.162328 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.162616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.162724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.162922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.163004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.163218 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.169793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.170076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.170212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 07:45:11.170358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.170497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.170637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.170802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.170946 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.171083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.171221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.171361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.171496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.171637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.171778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.171915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.172054 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.172202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.172339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.172476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.172611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.172762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.172888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.173033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.173153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.173312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.173431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.173581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.173858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 07:45:11.208057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e32ba3a9c0 pid: 81574 (default app) Sep 16 07:45:11.208578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e32ba3a9c0 pid: 81573 (default app) Sep 16 07:45:11.211330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1fafc-c6a0-4913-ae9a-718dbf7953e1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:11.211620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1fafc-c6a0-4913-ae9a-718dbf7953e1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:11.211897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1fafc-c6a0-4913-ae9a-718dbf7953e1 None None] PATH_INFO: `/v3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:11.216013 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1/1] 10.222.0.59 () {50 vars in 791 bytes} [Fri Sep 16 07:45:11 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:18.835298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd9fe-c2a2-496b-b6b1-23b3ea569a6b None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:18.835806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd9fe-c2a2-496b-b6b1-23b3ea569a6b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:18.835899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd9fe-c2a2-496b-b6b1-23b3ea569a6b None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:18.840093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1/2] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:45:18.845967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c7e9e-bd24-49be-b0bc-d0153c0e30de None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:18.846414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c7e9e-bd24-49be-b0bc-d0153c0e30de None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:18.846510 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c7e9e-bd24-49be-b0bc-d0153c0e30de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:18.916979 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: INFO dbcounter [None req-277c7e9e-bd24-49be-b0bc-d0153c0e30de None None] Registered counter for database keystone Sep 16 07:45:18.936407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG oslo_db.sqlalchemy.engines [None req-277c7e9e-bd24-49be-b0bc-d0153c0e30de None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81574) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 16 07:45:18.942871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writer thread running {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 16 07:45:19.121456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG passlib.handlers.bcrypt [None req-277c7e9e-bd24-49be-b0bc-d0153c0e30de None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81574) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 16 07:45:19.121769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG passlib.handlers.bcrypt [None req-277c7e9e-bd24-49be-b0bc-d0153c0e30de None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81574) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 16 07:45:19.179534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-277c7e9e-bd24-49be-b0bc-d0153c0e30de None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:19.181138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-277c7e9e-bd24-49be-b0bc-d0153c0e30de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:19.305961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 2/3] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 16 07:45:19.776779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bd9fe-c2a2-496b-b6b1-23b3ea569a6b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:19.846773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: INFO dbcounter [None req-ca5bd9fe-c2a2-496b-b6b1-23b3ea569a6b None None] Registered counter for database keystone Sep 16 07:45:19.868262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG oslo_db.sqlalchemy.engines [None req-ca5bd9fe-c2a2-496b-b6b1-23b3ea569a6b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81573) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 16 07:45:19.874456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writer thread running {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 16 07:45:19.993657 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bd9fe-c2a2-496b-b6b1-23b3ea569a6b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:19.997849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae19bd-dded-44a7-baf9-89535a44941a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:19.998475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae19bd-dded-44a7-baf9-89535a44941a admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:19.998787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae19bd-dded-44a7-baf9-89535a44941a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:19.999067 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae19bd-dded-44a7-baf9-89535a44941a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:20.007163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae19bd-dded-44a7-baf9-89535a44941a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:20.057869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5aae19bd-dded-44a7-baf9-89535a44941a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 16 07:45:20.058216 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae19bd-dded-44a7-baf9-89535a44941a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:20.067715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 2/4] 10.222.0.59 () {58 vars in 1324 bytes} [Fri Sep 16 07:45:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 07:45:20.073729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c7e9e-bd24-49be-b0bc-d0153c0e30de None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:20.090622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c7e9e-bd24-49be-b0bc-d0153c0e30de None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:20.095624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df2d4e-08fe-4fc8-a99d-8f16651e52f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:20.096727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df2d4e-08fe-4fc8-a99d-8f16651e52f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:20.097007 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df2d4e-08fe-4fc8-a99d-8f16651e52f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:20.097278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df2d4e-08fe-4fc8-a99d-8f16651e52f8 admin admin] PATH_INFO: `/v3/projects/1dfa255cacf3473b9f6619fec53b8cf1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:20.098867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df2d4e-08fe-4fc8-a99d-8f16651e52f8 admin admin] RBAC: Authorizing `identity:get_project(project_id=1dfa255cacf3473b9f6619fec53b8cf1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:20.151281 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: WARNING keystone.common.rbac_enforcer.enforcer [None req-17df2d4e-08fe-4fc8-a99d-8f16651e52f8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 16 07:45:20.151628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df2d4e-08fe-4fc8-a99d-8f16651e52f8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:20.156620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 3/5] 10.222.0.59 () {56 vars in 1207 bytes} [Fri Sep 16 07:45:20 2022] GET /identity/v3/projects/1dfa255cacf3473b9f6619fec53b8cf1 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:22.257469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3ed5b-9e54-4998-b0fe-8cfabbd800b5 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:22.257972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3ed5b-9e54-4998-b0fe-8cfabbd800b5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:22.258076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3ed5b-9e54-4998-b0fe-8cfabbd800b5 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:22.262076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 3/6] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:45:22.268014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e11b-b9ab-4c15-9750-9ec1c4ec3a47 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:22.268442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e11b-b9ab-4c15-9750-9ec1c4ec3a47 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:22.268574 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e11b-b9ab-4c15-9750-9ec1c4ec3a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:22.288462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0ce3e11b-b9ab-4c15-9750-9ec1c4ec3a47 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:22.289981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0ce3e11b-b9ab-4c15-9750-9ec1c4ec3a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:22.298215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 4/7] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 16 07:45:22.562738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3ed5b-9e54-4998-b0fe-8cfabbd800b5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:22.575003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3ed5b-9e54-4998-b0fe-8cfabbd800b5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:22.579196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d3b4d-0f70-4509-9e41-b8ff82faf225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:22.579793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d3b4d-0f70-4509-9e41-b8ff82faf225 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:22.580076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d3b4d-0f70-4509-9e41-b8ff82faf225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:22.580348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d3b4d-0f70-4509-9e41-b8ff82faf225 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:22.587948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d3b4d-0f70-4509-9e41-b8ff82faf225 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:22.589703 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d3b4d-0f70-4509-9e41-b8ff82faf225 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:22.598297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 4/8] 10.222.0.59 () {58 vars in 1324 bytes} [Fri Sep 16 07:45:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 07:45:22.603478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3e11b-b9ab-4c15-9750-9ec1c4ec3a47 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:22.612873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3e11b-b9ab-4c15-9750-9ec1c4ec3a47 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:22.617104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e221c-50f2-4cc0-b21a-e895971e37f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:22.617719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e221c-50f2-4cc0-b21a-e895971e37f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:22.617997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e221c-50f2-4cc0-b21a-e895971e37f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:22.618269 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e221c-50f2-4cc0-b21a-e895971e37f7 admin admin] PATH_INFO: `/v3/users/4763f3bdfee74904a41057be46f076ed` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:22.619611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e221c-50f2-4cc0-b21a-e895971e37f7 admin admin] RBAC: Authorizing `identity:get_user(user_id=4763f3bdfee74904a41057be46f076ed)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:22.621427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e221c-50f2-4cc0-b21a-e895971e37f7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:22.623559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 5/9] 10.222.0.59 () {56 vars in 1198 bytes} [Fri Sep 16 07:45:22 2022] GET /identity/v3/users/4763f3bdfee74904a41057be46f076ed => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:24.796605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061e3f-0ae9-4b31-aff8-49094204d9d2 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:24.797106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061e3f-0ae9-4b31-aff8-49094204d9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:24.797316 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061e3f-0ae9-4b31-aff8-49094204d9d2 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:24.798755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 5/10] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:24.803933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554071f6-a561-4098-a557-11f1055774a4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:24.804303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554071f6-a561-4098-a557-11f1055774a4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:24.804496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554071f6-a561-4098-a557-11f1055774a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:24.823143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-554071f6-a561-4098-a557-11f1055774a4 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:24.824330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-554071f6-a561-4098-a557-11f1055774a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:24.860328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817955e-f2dd-43d5-9c51-19c061e89721 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:24.860620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817955e-f2dd-43d5-9c51-19c061e89721 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:24.860901 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817955e-f2dd-43d5-9c51-19c061e89721 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:24.862216 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 6/11] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:45:24.867414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51cec2-8a2b-40b9-9416-24bb93fe3743 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:24.867696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51cec2-8a2b-40b9-9416-24bb93fe3743 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:24.868088 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51cec2-8a2b-40b9-9416-24bb93fe3743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:24.906009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 6/12] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:24.910401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG passlib.handlers.bcrypt [None req-7a51cec2-8a2b-40b9-9416-24bb93fe3743 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81573) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 16 07:45:24.910832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG passlib.handlers.bcrypt [None req-7a51cec2-8a2b-40b9-9416-24bb93fe3743 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81573) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 16 07:45:24.960690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7a51cec2-8a2b-40b9-9416-24bb93fe3743 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:24.962257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7a51cec2-8a2b-40b9-9416-24bb93fe3743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:25.060088 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 7/13] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:45:25.173915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554071f6-a561-4098-a557-11f1055774a4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:25.197588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:25.276167 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554071f6-a561-4098-a557-11f1055774a4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:25.308855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01364c71-d36b-4286-bc66-45c6034e6499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:25.309486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01364c71-d36b-4286-bc66-45c6034e6499 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:25.309768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01364c71-d36b-4286-bc66-45c6034e6499 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:25.310071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01364c71-d36b-4286-bc66-45c6034e6499 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:25.321627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51cec2-8a2b-40b9-9416-24bb93fe3743 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:25.321750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01364c71-d36b-4286-bc66-45c6034e6499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-01364c71-d36b-4286-bc66-45c6034e6499 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:25.330671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:25.331699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:45:25.332467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 7/14] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:45:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:25.339003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01364c71-d36b-4286-bc66-45c6034e6499 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:25.418363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:25.424662 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51cec2-8a2b-40b9-9416-24bb93fe3743 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:25.427647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:25.435882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01364c71-d36b-4286-bc66-45c6034e6499 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:25.461369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f15c6-1dea-4060-b2ca-cbd8772a507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:25.462229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f15c6-1dea-4060-b2ca-cbd8772a507c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:25.462512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f15c6-1dea-4060-b2ca-cbd8772a507c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:25.462819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f15c6-1dea-4060-b2ca-cbd8772a507c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:25.470441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f61a0d-37e0-45f2-bb97-6aa0b68110e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:25.471085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f61a0d-37e0-45f2-bb97-6aa0b68110e5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:25.471368 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f61a0d-37e0-45f2-bb97-6aa0b68110e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:25.471643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f61a0d-37e0-45f2-bb97-6aa0b68110e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:25.472310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f61a0d-37e0-45f2-bb97-6aa0b68110e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:25.472616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f61a0d-37e0-45f2-bb97-6aa0b68110e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:25.474018 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f61a0d-37e0-45f2-bb97-6aa0b68110e5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:25.485605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 8/15] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:25.491237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f61a0d-37e0-45f2-bb97-6aa0b68110e5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:25.530462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f15c6-1dea-4060-b2ca-cbd8772a507c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:25.532251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f15c6-1dea-4060-b2ca-cbd8772a507c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:25.586507 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f61a0d-37e0-45f2-bb97-6aa0b68110e5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:25.618944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111f8df-ae56-41a4-9821-bc0c621f5eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:25.619560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f8df-ae56-41a4-9821-bc0c621f5eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:25.619841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f8df-ae56-41a4-9821-bc0c621f5eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:25.620112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f8df-ae56-41a4-9821-bc0c621f5eb3 None admin] PATH_INFO: `/v3/users/4763f3bdfee74904a41057be46f076ed` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:25.621288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111f8df-ae56-41a4-9821-bc0c621f5eb3 None admin] RBAC: Authorizing `identity:get_user(user_id=4763f3bdfee74904a41057be46f076ed)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:25.622783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111f8df-ae56-41a4-9821-bc0c621f5eb3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:25.624337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 9/16] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:25 2022] GET /identity/v3/users/4763f3bdfee74904a41057be46f076ed => generated 247 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:25.627131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 8/17] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 07:45:25.629877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111f8df-ae56-41a4-9821-bc0c621f5eb3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:25.632355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f15c6-1dea-4060-b2ca-cbd8772a507c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:25.635023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:25.645110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:25.730556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111f8df-ae56-41a4-9821-bc0c621f5eb3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:25.730846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f15c6-1dea-4060-b2ca-cbd8772a507c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:25.765093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4230d85-4d9f-406f-ab8c-6fdecd22c392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:25.765294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c9ede-9211-4dc4-8f29-8e38451b7a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:25.765741 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230d85-4d9f-406f-ab8c-6fdecd22c392 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:25.766026 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230d85-4d9f-406f-ab8c-6fdecd22c392 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:25.766140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c9ede-9211-4dc4-8f29-8e38451b7a89 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:25.766305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230d85-4d9f-406f-ab8c-6fdecd22c392 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:25.766419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c9ede-9211-4dc4-8f29-8e38451b7a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:25.766715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c9ede-9211-4dc4-8f29-8e38451b7a89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:25.767475 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4230d85-4d9f-406f-ab8c-6fdecd22c392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:25.768943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4230d85-4d9f-406f-ab8c-6fdecd22c392 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:25.770490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 10/18] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:25.776143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4230d85-4d9f-406f-ab8c-6fdecd22c392 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:25.785140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c9ede-9211-4dc4-8f29-8e38451b7a89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:25.786603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c9ede-9211-4dc4-8f29-8e38451b7a89 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-8b9c9ede-9211-4dc4-8f29-8e38451b7a89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:45:25.804363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:25.805838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:45:25.806779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:45:25.806779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:45:25.806779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:25.806779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 9/19] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:45:25.811781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c9ede-9211-4dc4-8f29-8e38451b7a89 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:25.857065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:25.869081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:25.875406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4230d85-4d9f-406f-ab8c-6fdecd22c392 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:25.909798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72660e1a-c339-436e-af43-b7fae0ff2a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:25.910404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72660e1a-c339-436e-af43-b7fae0ff2a97 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:25.910724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72660e1a-c339-436e-af43-b7fae0ff2a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:25.910982 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72660e1a-c339-436e-af43-b7fae0ff2a97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:25.911704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72660e1a-c339-436e-af43-b7fae0ff2a97 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.domain.id=default, user.id=4763f3bdfee74904a41057be46f076ed) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:25.912010 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72660e1a-c339-436e-af43-b7fae0ff2a97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:25.912911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c9ede-9211-4dc4-8f29-8e38451b7a89 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:25.913457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72660e1a-c339-436e-af43-b7fae0ff2a97 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:25.920075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 11/20] 10.222.0.59 () {56 vars in 1322 bytes} [Fri Sep 16 07:45:25 2022] GET /identity/v3/role_assignments?user.id=4763f3bdfee74904a41057be46f076ed&scope.domain.id=default&role.id=62eb3e525077429aacc0ada63300802e => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:25.947838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab0463-dc6a-454c-9ba4-81fdc93a1064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:25.948471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab0463-dc6a-454c-9ba4-81fdc93a1064 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:25.948750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab0463-dc6a-454c-9ba4-81fdc93a1064 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:25.949021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab0463-dc6a-454c-9ba4-81fdc93a1064 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:25.949680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab0463-dc6a-454c-9ba4-81fdc93a1064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:25.949985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab0463-dc6a-454c-9ba4-81fdc93a1064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:25.952036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab0463-dc6a-454c-9ba4-81fdc93a1064 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:25.970876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 10/21] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:28.123519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d42ad-f31d-4248-88fd-8eeb752b6dc7 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:28.124558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d42ad-f31d-4248-88fd-8eeb752b6dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:28.125046 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d42ad-f31d-4248-88fd-8eeb752b6dc7 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:28.126588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 12/22] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:28.132324 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62852-804c-4541-8f7a-2cbda68ad93d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:28.133181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62852-804c-4541-8f7a-2cbda68ad93d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:28.133662 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62852-804c-4541-8f7a-2cbda68ad93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:28.159303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cea62852-804c-4541-8f7a-2cbda68ad93d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:28.160909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cea62852-804c-4541-8f7a-2cbda68ad93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:28.242240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 11/23] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:28.770516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d42ad-f31d-4248-88fd-8eeb752b6dc7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:28.852843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766720d-bcb2-4714-bd5d-93af595dd49c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:28.853159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766720d-bcb2-4714-bd5d-93af595dd49c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:28.853396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766720d-bcb2-4714-bd5d-93af595dd49c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:28.854819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 12/24] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:28.860953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172875f-3ff9-4464-b712-9285f1735037 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:28.861234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172875f-3ff9-4464-b712-9285f1735037 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:28.861509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172875f-3ff9-4464-b712-9285f1735037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:28.900773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6172875f-3ff9-4464-b712-9285f1735037 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:28.902000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6172875f-3ff9-4464-b712-9285f1735037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:28.960887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d42ad-f31d-4248-88fd-8eeb752b6dc7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:29.018940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:29.043893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0058d-397e-4d55-843e-05732ba56251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:29.045059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0058d-397e-4d55-843e-05732ba56251 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:29.045335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0058d-397e-4d55-843e-05732ba56251 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:29.045606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0058d-397e-4d55-843e-05732ba56251 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:29.061582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 13/25] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:45:29.063369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf74136-0f04-49c7-885a-6cbbaadfceba None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:29.063811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf74136-0f04-49c7-885a-6cbbaadfceba None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:29.064234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf74136-0f04-49c7-885a-6cbbaadfceba None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:29.066211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 14/26] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:29.071998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0058d-397e-4d55-843e-05732ba56251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:29.072247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6c0eb-8cd8-4124-a80d-a3a08f8d97c9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:29.072671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6c0eb-8cd8-4124-a80d-a3a08f8d97c9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:29.073094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6c0eb-8cd8-4124-a80d-a3a08f8d97c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1cc0058d-397e-4d55-843e-05732ba56251 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:29.077579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:29.078752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:45:29.079720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 13/27] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:45:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:29.087208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0058d-397e-4d55-843e-05732ba56251 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:29.095503 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:29.096484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-faa6c0eb-8cd8-4124-a80d-a3a08f8d97c9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:29.097869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-faa6c0eb-8cd8-4124-a80d-a3a08f8d97c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:29.172028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 15/28] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:45:29.223956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0058d-397e-4d55-843e-05732ba56251 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:29.228634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88bb94-ca79-41ec-9b4d-ba06dda54163 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:29.229100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88bb94-ca79-41ec-9b4d-ba06dda54163 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:29.229516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88bb94-ca79-41ec-9b4d-ba06dda54163 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:29.230998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 16/29] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:45:29.241009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd0d11-4aa8-4da7-99ce-28f07e4e62ab None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:29.241471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd0d11-4aa8-4da7-99ce-28f07e4e62ab None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:29.244939 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd0d11-4aa8-4da7-99ce-28f07e4e62ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:29.276066 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cbcd0d11-4aa8-4da7-99ce-28f07e4e62ab None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:29.277465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cbcd0d11-4aa8-4da7-99ce-28f07e4e62ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:29.290855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8377b58-9822-4a5b-a390-ad756b07aee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:29.290855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8377b58-9822-4a5b-a390-ad756b07aee0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:29.290855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8377b58-9822-4a5b-a390-ad756b07aee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:29.290855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8377b58-9822-4a5b-a390-ad756b07aee0 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:29.291254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8377b58-9822-4a5b-a390-ad756b07aee0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:29.291254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8377b58-9822-4a5b-a390-ad756b07aee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:29.291254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8377b58-9822-4a5b-a390-ad756b07aee0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:29.303536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 14/30] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:29.311155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8377b58-9822-4a5b-a390-ad756b07aee0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:29.416408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:29.441622 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 17/31] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:29.453279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd0d11-4aa8-4da7-99ce-28f07e4e62ab None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:29.478946 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8377b58-9822-4a5b-a390-ad756b07aee0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:29.527780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:29.543647 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c848c-dc11-4360-be32-4029bb9be3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:29.544975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c848c-dc11-4360-be32-4029bb9be3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:29.545227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c848c-dc11-4360-be32-4029bb9be3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:29.545528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c848c-dc11-4360-be32-4029bb9be3f2 None admin] PATH_INFO: `/v3/users/4763f3bdfee74904a41057be46f076ed` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:29.546721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c848c-dc11-4360-be32-4029bb9be3f2 None admin] RBAC: Authorizing `identity:get_user(user_id=4763f3bdfee74904a41057be46f076ed)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:29.548203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c848c-dc11-4360-be32-4029bb9be3f2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:29.549873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 15/32] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:29 2022] GET /identity/v3/users/4763f3bdfee74904a41057be46f076ed => generated 247 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:29.555469 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c848c-dc11-4360-be32-4029bb9be3f2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:29.688880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd0d11-4aa8-4da7-99ce-28f07e4e62ab None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:29.705343 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c848c-dc11-4360-be32-4029bb9be3f2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:29.757041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c031563-3629-42ce-92f3-1052f1586c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:29.757682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031563-3629-42ce-92f3-1052f1586c32 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:29.757961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031563-3629-42ce-92f3-1052f1586c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:29.758235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031563-3629-42ce-92f3-1052f1586c32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:29.773846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c031563-3629-42ce-92f3-1052f1586c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:29.775339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c031563-3629-42ce-92f3-1052f1586c32 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:29.776899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a54580-4f77-4b9d-a736-b8ffd5e16e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:29.777545 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a54580-4f77-4b9d-a736-b8ffd5e16e1d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:29.777879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a54580-4f77-4b9d-a736-b8ffd5e16e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:29.778097 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a54580-4f77-4b9d-a736-b8ffd5e16e1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:29.779830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a54580-4f77-4b9d-a736-b8ffd5e16e1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:29.781263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a54580-4f77-4b9d-a736-b8ffd5e16e1d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:29.784040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 16/33] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:29.784775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a54580-4f77-4b9d-a736-b8ffd5e16e1d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:29.795101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-6c031563-3629-42ce-92f3-1052f1586c32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:45:29.814240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:29.821112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:45:29.821904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:45:29.821904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:45:29.821904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:29.821904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 18/34] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:29.821904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c031563-3629-42ce-92f3-1052f1586c32 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:29.915246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a54580-4f77-4b9d-a736-b8ffd5e16e1d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:29.932417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:29.948773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c031563-3629-42ce-92f3-1052f1586c32 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:29.954371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ac3ff-54a2-42f2-a53d-6d2a60d5fa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:29.957594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ac3ff-54a2-42f2-a53d-6d2a60d5fa67 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:29.957594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ac3ff-54a2-42f2-a53d-6d2a60d5fa67 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:29.957594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ac3ff-54a2-42f2-a53d-6d2a60d5fa67 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:29.957594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ac3ff-54a2-42f2-a53d-6d2a60d5fa67 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:29.957922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ac3ff-54a2-42f2-a53d-6d2a60d5fa67 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:29.988664 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2f203-10b1-4c3c-a435-c1cc1e69c447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:29.989354 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2f203-10b1-4c3c-a435-c1cc1e69c447 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:29.989635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2f203-10b1-4c3c-a435-c1cc1e69c447 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:29.992255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2f203-10b1-4c3c-a435-c1cc1e69c447 None admin] PATH_INFO: `/v3/domains/default/users/4763f3bdfee74904a41057be46f076ed/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:29.992255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2f203-10b1-4c3c-a435-c1cc1e69c447 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4763f3bdfee74904a41057be46f076ed, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:29.993681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2f203-10b1-4c3c-a435-c1cc1e69c447 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:30.155417 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 17/35] 10.222.0.59 () {60 vars in 1131 bytes} [Fri Sep 16 07:45:29 2022] POST /identity/v3/roles => generated 225 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:45:30.156112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ac3ff-54a2-42f2-a53d-6d2a60d5fa67 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:30.219877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 19/36] 10.222.0.59 () {58 vars in 1361 bytes} [Fri Sep 16 07:45:29 2022] PUT /identity/v3/domains/default/users/4763f3bdfee74904a41057be46f076ed/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:45:30.222156 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2f203-10b1-4c3c-a435-c1cc1e69c447 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:30.233313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:30.329177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ac3ff-54a2-42f2-a53d-6d2a60d5fa67 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:30.384830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa329e-47f0-4191-995b-f0395c9f1233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:30.385601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa329e-47f0-4191-995b-f0395c9f1233 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:30.385888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa329e-47f0-4191-995b-f0395c9f1233 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:30.386209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa329e-47f0-4191-995b-f0395c9f1233 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:30.387039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa329e-47f0-4191-995b-f0395c9f1233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:30.387349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa329e-47f0-4191-995b-f0395c9f1233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:30.389489 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa329e-47f0-4191-995b-f0395c9f1233 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:30.414171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2f203-10b1-4c3c-a435-c1cc1e69c447 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:30.417780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 18/37] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:30.419432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4214-7a99-4b32-84ea-1fe82520e89b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:30.419722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4214-7a99-4b32-84ea-1fe82520e89b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:30.420000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4214-7a99-4b32-84ea-1fe82520e89b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:30.425805 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 19/38] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:30.430969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11345d6-ef13-4968-a319-cf6c64b10d9a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:30.431259 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11345d6-ef13-4968-a319-cf6c64b10d9a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:30.431542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11345d6-ef13-4968-a319-cf6c64b10d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:30.458815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f11345d6-ef13-4968-a319-cf6c64b10d9a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:30.460451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f11345d6-ef13-4968-a319-cf6c64b10d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:30.469195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73936da6-ad21-429b-a86f-be7c2cee79c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:30.470198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936da6-ad21-429b-a86f-be7c2cee79c5 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:30.470751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936da6-ad21-429b-a86f-be7c2cee79c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:30.471190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936da6-ad21-429b-a86f-be7c2cee79c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:30.472032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73936da6-ad21-429b-a86f-be7c2cee79c5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:30.473673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73936da6-ad21-429b-a86f-be7c2cee79c5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:30.542633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 20/39] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:30.543547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11345d6-ef13-4968-a319-cf6c64b10d9a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:30.550272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 20/40] 10.222.0.59 () {60 vars in 1131 bytes} [Fri Sep 16 07:45:30 2022] POST /identity/v3/roles => generated 219 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:45:30.601395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aafaed-e6d9-49ee-baf8-bc57fe59d804 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:30.601747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aafaed-e6d9-49ee-baf8-bc57fe59d804 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:30.601963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aafaed-e6d9-49ee-baf8-bc57fe59d804 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:30.603370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 21/41] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:45:30.609811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bb651-1e47-45e6-9db5-266f308c35db None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:30.610212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bb651-1e47-45e6-9db5-266f308c35db None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:30.610594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bb651-1e47-45e6-9db5-266f308c35db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:30.611554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:30.633936 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f44bb651-1e47-45e6-9db5-266f308c35db None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:30.635907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f44bb651-1e47-45e6-9db5-266f308c35db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:30.645722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:30.718010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 22/42] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:45:30.745082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11345d6-ef13-4968-a319-cf6c64b10d9a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:30.787075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3b03b-0cbb-4bdb-98de-c3588df5bae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:30.787701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b03b-0cbb-4bdb-98de-c3588df5bae0 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:30.787988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b03b-0cbb-4bdb-98de-c3588df5bae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:30.788269 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3b03b-0cbb-4bdb-98de-c3588df5bae0 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:30.788933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3b03b-0cbb-4bdb-98de-c3588df5bae0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:30.790476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3b03b-0cbb-4bdb-98de-c3588df5bae0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:30.900953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 21/43] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:45:30 2022] POST /identity/v3/projects => generated 301 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:45:30.906790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bb651-1e47-45e6-9db5-266f308c35db None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:31.017787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bb651-1e47-45e6-9db5-266f308c35db None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:31.018211 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3b03b-0cbb-4bdb-98de-c3588df5bae0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:31.056572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd471a7-b8ee-44bd-a2f6-4942ae7a823c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:31.057203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd471a7-b8ee-44bd-a2f6-4942ae7a823c None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:31.057483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd471a7-b8ee-44bd-a2f6-4942ae7a823c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:31.057757 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd471a7-b8ee-44bd-a2f6-4942ae7a823c None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:31.058434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd471a7-b8ee-44bd-a2f6-4942ae7a823c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:31.059950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd471a7-b8ee-44bd-a2f6-4942ae7a823c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:31.162310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:31.168559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 23/44] 10.222.0.59 () {60 vars in 1131 bytes} [Fri Sep 16 07:45:30 2022] POST /identity/v3/roles => generated 223 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:45:31.254020 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3b03b-0cbb-4bdb-98de-c3588df5bae0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:31.258143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:31.293799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9798d9-3071-4054-a165-0a77ef1dc319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:31.295519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9798d9-3071-4054-a165-0a77ef1dc319 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:31.296392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9798d9-3071-4054-a165-0a77ef1dc319 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:31.296668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9798d9-3071-4054-a165-0a77ef1dc319 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:31.298083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9798d9-3071-4054-a165-0a77ef1dc319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:31.299597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9798d9-3071-4054-a165-0a77ef1dc319 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:31.301263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 22/45] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:31.307189 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd471a7-b8ee-44bd-a2f6-4942ae7a823c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:31.412976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd471a7-b8ee-44bd-a2f6-4942ae7a823c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:31.446185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35be86-ab75-4157-9284-bef0795bc308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:31.446828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35be86-ab75-4157-9284-bef0795bc308 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:31.447110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35be86-ab75-4157-9284-bef0795bc308 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:31.447384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35be86-ab75-4157-9284-bef0795bc308 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:31.448023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35be86-ab75-4157-9284-bef0795bc308 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:31.449502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35be86-ab75-4157-9284-bef0795bc308 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:31.573438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 24/46] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:45:31 2022] POST /identity/v3/projects => generated 312 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:45:32.545066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea61225-ec47-464a-bddc-50057244cf8d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:32.545621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea61225-ec47-464a-bddc-50057244cf8d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:32.545714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea61225-ec47-464a-bddc-50057244cf8d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:32.547088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 23/47] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:32.552428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79acfe3-59c2-4620-94db-22df817c7328 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:32.552816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79acfe3-59c2-4620-94db-22df817c7328 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:32.552995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79acfe3-59c2-4620-94db-22df817c7328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:32.570899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a79acfe3-59c2-4620-94db-22df817c7328 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:32.572100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a79acfe3-59c2-4620-94db-22df817c7328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:32.616755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:32.636827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 25/48] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:32.903872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea61225-ec47-464a-bddc-50057244cf8d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:33.002906 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea61225-ec47-464a-bddc-50057244cf8d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:33.035317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e8e46-c20c-42c4-a71d-9d3961c53fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:33.035947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e8e46-c20c-42c4-a71d-9d3961c53fbe None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:33.036229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e8e46-c20c-42c4-a71d-9d3961c53fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:33.036503 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e8e46-c20c-42c4-a71d-9d3961c53fbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:33.047264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e8e46-c20c-42c4-a71d-9d3961c53fbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-000e8e46-c20c-42c4-a71d-9d3961c53fbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:33.052176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:33.052997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:45:33.053772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 24/49] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:45:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:33.059187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79acfe3-59c2-4620-94db-22df817c7328 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:33.155997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79acfe3-59c2-4620-94db-22df817c7328 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:33.189255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a81224-a4b8-4417-9ba1-6d737a25c11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:33.190140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a81224-a4b8-4417-9ba1-6d737a25c11f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:33.190425 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a81224-a4b8-4417-9ba1-6d737a25c11f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:33.190724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a81224-a4b8-4417-9ba1-6d737a25c11f None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:33.191399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a81224-a4b8-4417-9ba1-6d737a25c11f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:33.191710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a81224-a4b8-4417-9ba1-6d737a25c11f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:33.193099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a81224-a4b8-4417-9ba1-6d737a25c11f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:33.205202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 26/50] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:33.211566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e8e46-c20c-42c4-a71d-9d3961c53fbe None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:33.241586 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:33.307977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e8e46-c20c-42c4-a71d-9d3961c53fbe None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:33.341856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a9047-825f-411d-84d0-ee210ba899e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:33.342486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a9047-825f-411d-84d0-ee210ba899e3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:33.342796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a9047-825f-411d-84d0-ee210ba899e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:33.343077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a9047-825f-411d-84d0-ee210ba899e3 None admin] PATH_INFO: `/v3/users/4763f3bdfee74904a41057be46f076ed` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:33.344211 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a9047-825f-411d-84d0-ee210ba899e3 None admin] RBAC: Authorizing `identity:get_user(user_id=4763f3bdfee74904a41057be46f076ed)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:33.345692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a9047-825f-411d-84d0-ee210ba899e3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:33.347309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 25/51] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:33 2022] GET /identity/v3/users/4763f3bdfee74904a41057be46f076ed => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:33.352851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a81224-a4b8-4417-9ba1-6d737a25c11f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:33.395640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:33.451746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a81224-a4b8-4417-9ba1-6d737a25c11f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:33.484441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e31db2-e6a2-4b54-99b9-08f2f0a561cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:33.485072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31db2-e6a2-4b54-99b9-08f2f0a561cb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:33.485351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31db2-e6a2-4b54-99b9-08f2f0a561cb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:33.485625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e31db2-e6a2-4b54-99b9-08f2f0a561cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:33.486750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e31db2-e6a2-4b54-99b9-08f2f0a561cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:33.488191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e31db2-e6a2-4b54-99b9-08f2f0a561cb None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:33.489750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 27/52] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:33.495511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a9047-825f-411d-84d0-ee210ba899e3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:33.597560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:33.600648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a9047-825f-411d-84d0-ee210ba899e3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:33.635834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e96a3-11dd-4246-b277-e79c80c5da47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:33.636441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e96a3-11dd-4246-b277-e79c80c5da47 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:33.636723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e96a3-11dd-4246-b277-e79c80c5da47 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:33.637000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e96a3-11dd-4246-b277-e79c80c5da47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:33.637724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e96a3-11dd-4246-b277-e79c80c5da47 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.domain.id=default, user.id=4763f3bdfee74904a41057be46f076ed) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:33.638069 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e96a3-11dd-4246-b277-e79c80c5da47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:33.639551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e96a3-11dd-4246-b277-e79c80c5da47 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:33.645204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 26/53] 10.222.0.59 () {56 vars in 1322 bytes} [Fri Sep 16 07:45:33 2022] GET /identity/v3/role_assignments?user.id=4763f3bdfee74904a41057be46f076ed&scope.domain.id=default&role.id=62eb3e525077429aacc0ada63300802e => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:33.693361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264148b-0c22-4c13-9ee8-54c6b0ca8133 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:33.693883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264148b-0c22-4c13-9ee8-54c6b0ca8133 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:33.694062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264148b-0c22-4c13-9ee8-54c6b0ca8133 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:33.695555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 28/54] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:33.701590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b0e2-f75f-46af-b3aa-553dd677703d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:33.701925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b0e2-f75f-46af-b3aa-553dd677703d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:33.702146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b0e2-f75f-46af-b3aa-553dd677703d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:33.721175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1141b0e2-f75f-46af-b3aa-553dd677703d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:33.722441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1141b0e2-f75f-46af-b3aa-553dd677703d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:33.794969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 27/55] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:45:34.057033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264148b-0c22-4c13-9ee8-54c6b0ca8133 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:34.258838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264148b-0c22-4c13-9ee8-54c6b0ca8133 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:34.268963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:34.291585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb92aa-3e3a-4e6e-ad6a-0c3106787dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:34.292213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb92aa-3e3a-4e6e-ad6a-0c3106787dde None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:34.292496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb92aa-3e3a-4e6e-ad6a-0c3106787dde None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:34.292771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb92aa-3e3a-4e6e-ad6a-0c3106787dde None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:34.293878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb92aa-3e3a-4e6e-ad6a-0c3106787dde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:34.296036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb92aa-3e3a-4e6e-ad6a-0c3106787dde None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:34.297709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 29/56] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:34.303331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141b0e2-f75f-46af-b3aa-553dd677703d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:34.391484 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:34.397739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141b0e2-f75f-46af-b3aa-553dd677703d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:34.430789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0657f-3457-43ce-a3df-928619833d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:34.431390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0657f-3457-43ce-a3df-928619833d27 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:34.431668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0657f-3457-43ce-a3df-928619833d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:34.431939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0657f-3457-43ce-a3df-928619833d27 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:34.432578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0657f-3457-43ce-a3df-928619833d27 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:34.434024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0657f-3457-43ce-a3df-928619833d27 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:34.526849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 28/57] 10.222.0.59 () {60 vars in 1140 bytes} [Fri Sep 16 07:45:34 2022] POST /identity/v3/projects => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:45:36.650785 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aad06-0d8f-419e-8e30-1a18640fbb96 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:36.651300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aad06-0d8f-419e-8e30-1a18640fbb96 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:36.651382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aad06-0d8f-419e-8e30-1a18640fbb96 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:36.652667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 30/58] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:36.657925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43055e-85fc-45ca-9b6e-7f075bb9981a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:36.658290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43055e-85fc-45ca-9b6e-7f075bb9981a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:36.658479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43055e-85fc-45ca-9b6e-7f075bb9981a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:36.676574 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1b43055e-85fc-45ca-9b6e-7f075bb9981a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:36.677789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1b43055e-85fc-45ca-9b6e-7f075bb9981a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:36.741653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 29/59] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:37.006148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aad06-0d8f-419e-8e30-1a18640fbb96 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:37.104365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aad06-0d8f-419e-8e30-1a18640fbb96 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:37.136732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23622d-14c1-4966-b8c6-e233236a6c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:37.137358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23622d-14c1-4966-b8c6-e233236a6c56 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:37.137641 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23622d-14c1-4966-b8c6-e233236a6c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:37.137918 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23622d-14c1-4966-b8c6-e233236a6c56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:37.139053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23622d-14c1-4966-b8c6-e233236a6c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:37.140496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23622d-14c1-4966-b8c6-e233236a6c56 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:37.142033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 31/60] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:37.147542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43055e-85fc-45ca-9b6e-7f075bb9981a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:37.220578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:37.242226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43055e-85fc-45ca-9b6e-7f075bb9981a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:37.273925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2eac7e-288d-4761-833b-77769f40a1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:37.274476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eac7e-288d-4761-833b-77769f40a1ee None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:37.274958 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eac7e-288d-4761-833b-77769f40a1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:37.275277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eac7e-288d-4761-833b-77769f40a1ee None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:37.276066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2eac7e-288d-4761-833b-77769f40a1ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:37.277936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2eac7e-288d-4761-833b-77769f40a1ee None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:37.360379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 30/61] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:45:37 2022] POST /identity/v3/users => generated 275 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:45:40.665141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d13c94-7250-49bb-83e3-a76e6a7685ea None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:40.666157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d13c94-7250-49bb-83e3-a76e6a7685ea None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:40.666659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d13c94-7250-49bb-83e3-a76e6a7685ea None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:40.668492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 32/62] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:40.673595 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b5073-1138-4dda-902c-08f2d909dd44 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:40.674431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b5073-1138-4dda-902c-08f2d909dd44 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:40.674936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b5073-1138-4dda-902c-08f2d909dd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:40.703698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-794b5073-1138-4dda-902c-08f2d909dd44 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:40.705347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-794b5073-1138-4dda-902c-08f2d909dd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:40.808036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 31/63] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:40.908469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464d232-1612-4d3d-9abc-0d4b0c459e15 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:40.909158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464d232-1612-4d3d-9abc-0d4b0c459e15 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:40.909646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464d232-1612-4d3d-9abc-0d4b0c459e15 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:40.911231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 33/64] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:40.920189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f96416-4515-46ea-b64b-cfec7afd714a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:40.920740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f96416-4515-46ea-b64b-cfec7afd714a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:40.921211 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f96416-4515-46ea-b64b-cfec7afd714a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:40.940222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f6ca0-24d4-4b14-8388-0b9c82d59aa3 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:40.940855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f6ca0-24d4-4b14-8388-0b9c82d59aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:40.941332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f6ca0-24d4-4b14-8388-0b9c82d59aa3 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:40.943295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 34/65] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:40.949465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cf58d-6ff2-47ab-8603-38e258a600c7 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:40.950628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cf58d-6ff2-47ab-8603-38e258a600c7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:40.951180 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cf58d-6ff2-47ab-8603-38e258a600c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:40.955526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d7f96416-4515-46ea-b64b-cfec7afd714a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:40.956891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d7f96416-4515-46ea-b64b-cfec7afd714a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:40.982040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-605cf58d-6ff2-47ab-8603-38e258a600c7 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:40.983286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-605cf58d-6ff2-47ab-8603-38e258a600c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:41.045430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:41.063060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:41.065753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 32/66] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:45:41.068650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ac07a-986b-4301-8aef-91f4ca49f984 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:41.068936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ac07a-986b-4301-8aef-91f4ca49f984 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:41.069218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ac07a-986b-4301-8aef-91f4ca49f984 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:41.070659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 33/67] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:41.076169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72745028-4af7-417d-bdda-7bcfd944484d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:41.076454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72745028-4af7-417d-bdda-7bcfd944484d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:41.076731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72745028-4af7-417d-bdda-7bcfd944484d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:41.109944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-72745028-4af7-417d-bdda-7bcfd944484d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:41.113081 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-72745028-4af7-417d-bdda-7bcfd944484d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:41.117405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 35/68] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:45:41.174083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cf58d-6ff2-47ab-8603-38e258a600c7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:41.230664 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 34/69] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:41.327676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cf58d-6ff2-47ab-8603-38e258a600c7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:41.348285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72745028-4af7-417d-bdda-7bcfd944484d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:41.395121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e15fa8-8ea3-43af-a00c-bc1ce70ae052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:41.395754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e15fa8-8ea3-43af-a00c-bc1ce70ae052 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:41.396028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e15fa8-8ea3-43af-a00c-bc1ce70ae052 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:41.396303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e15fa8-8ea3-43af-a00c-bc1ce70ae052 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:41.409077 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e15fa8-8ea3-43af-a00c-bc1ce70ae052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d2e15fa8-8ea3-43af-a00c-bc1ce70ae052 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:41.415524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:41.416385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:41.417169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 36/70] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:41.423128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57793117-9cb2-4135-b9a2-39bd6391d3c6 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:41.423423 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57793117-9cb2-4135-b9a2-39bd6391d3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:41.423707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57793117-9cb2-4135-b9a2-39bd6391d3c6 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:41.425026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 37/71] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:41.426238 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57793117-9cb2-4135-b9a2-39bd6391d3c6 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:41.468341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:41.481311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72745028-4af7-417d-bdda-7bcfd944484d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:41.505756 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:41.518344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a40b01-a9c5-47c6-a644-10d27c3f2429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:41.519009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a40b01-a9c5-47c6-a644-10d27c3f2429 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:41.519296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a40b01-a9c5-47c6-a644-10d27c3f2429 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:41.519573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a40b01-a9c5-47c6-a644-10d27c3f2429 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:41.532555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a40b01-a9c5-47c6-a644-10d27c3f2429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f2a40b01-a9c5-47c6-a644-10d27c3f2429 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:41.537849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:41.538891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:41.539788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:45:41.540542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57793117-9cb2-4135-b9a2-39bd6391d3c6 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:41.540612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 35/72] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:45:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:45:41.540848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bd5d0-15a0-47ed-ac92-5f23a878c27b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:41.541108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bd5d0-15a0-47ed-ac92-5f23a878c27b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:41.541438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bd5d0-15a0-47ed-ac92-5f23a878c27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:41.559572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-659bd5d0-15a0-47ed-ac92-5f23a878c27b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:41.560775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-659bd5d0-15a0-47ed-ac92-5f23a878c27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:41.576549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a3384-3591-4e15-bdbc-cb03d0a34fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:41.577258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a3384-3591-4e15-bdbc-cb03d0a34fac None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:41.577608 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a3384-3591-4e15-bdbc-cb03d0a34fac None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:41.577954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a3384-3591-4e15-bdbc-cb03d0a34fac None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:41.578621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a3384-3591-4e15-bdbc-cb03d0a34fac None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:41.578955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a3384-3591-4e15-bdbc-cb03d0a34fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:41.580336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a3384-3591-4e15-bdbc-cb03d0a34fac None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:41.588327 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 38/73] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:45:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:41.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a3384-3591-4e15-bdbc-cb03d0a34fac None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:41.632309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 36/74] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:41.634286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bd5d0-15a0-47ed-ac92-5f23a878c27b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:41.689235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:41.694178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a3384-3591-4e15-bdbc-cb03d0a34fac None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:41.732447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45a9f8-bb06-4f4b-8b53-78c11f4ebc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:41.733119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45a9f8-bb06-4f4b-8b53-78c11f4ebc93 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:41.733403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45a9f8-bb06-4f4b-8b53-78c11f4ebc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:41.733678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45a9f8-bb06-4f4b-8b53-78c11f4ebc93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:41.744321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45a9f8-bb06-4f4b-8b53-78c11f4ebc93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:41.747090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bd5d0-15a0-47ed-ac92-5f23a878c27b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-1c45a9f8-bb06-4f4b-8b53-78c11f4ebc93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:41.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:41.751496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 39/75] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:45:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:41.757601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45a9f8-bb06-4f4b-8b53-78c11f4ebc93 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:41.760175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:41.781930 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748db08-6992-4933-8ba3-aa052c804d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:41.782580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748db08-6992-4933-8ba3-aa052c804d66 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:41.782897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748db08-6992-4933-8ba3-aa052c804d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:41.783179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748db08-6992-4933-8ba3-aa052c804d66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:41.795712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748db08-6992-4933-8ba3-aa052c804d66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7748db08-6992-4933-8ba3-aa052c804d66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:41.801651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:41.802559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:45:41.803362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 37/76] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:41.803994 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748db08-6992-4933-8ba3-aa052c804d66 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:41.871132 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45a9f8-bb06-4f4b-8b53-78c11f4ebc93 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:41.906493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513143c2-9db8-4756-b5b3-91779299ceec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:41.907152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513143c2-9db8-4756-b5b3-91779299ceec None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:41.907417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513143c2-9db8-4756-b5b3-91779299ceec None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:41.907517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748db08-6992-4933-8ba3-aa052c804d66 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:41.907691 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513143c2-9db8-4756-b5b3-91779299ceec None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:41.908356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513143c2-9db8-4756-b5b3-91779299ceec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:41.908666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513143c2-9db8-4756-b5b3-91779299ceec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:41.910047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513143c2-9db8-4756-b5b3-91779299ceec None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:41.917934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 40/77] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:41.922624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513143c2-9db8-4756-b5b3-91779299ceec None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:41.940523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b027ef-5ec7-46fc-b64c-d994aee49798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:41.941083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b027ef-5ec7-46fc-b64c-d994aee49798 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:41.941362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b027ef-5ec7-46fc-b64c-d994aee49798 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:41.941632 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b027ef-5ec7-46fc-b64c-d994aee49798 None admin] PATH_INFO: `/v3/users/71f957f6c8724676af13c99ddc4b62e3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:41.943905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:41.955335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b027ef-5ec7-46fc-b64c-d994aee49798 None admin] RBAC: Authorizing `identity:get_user(user_id=71f957f6c8724676af13c99ddc4b62e3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:41.956895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b027ef-5ec7-46fc-b64c-d994aee49798 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:41.958532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 38/78] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:41 2022] GET /identity/v3/users/71f957f6c8724676af13c99ddc4b62e3 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:41.959365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b027ef-5ec7-46fc-b64c-d994aee49798 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:41.988706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:42.022013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513143c2-9db8-4756-b5b3-91779299ceec None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:42.055271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c51c89-aa4a-4fc6-a177-8bdcc0bc0b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:42.055876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c51c89-aa4a-4fc6-a177-8bdcc0bc0b57 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:42.056160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c51c89-aa4a-4fc6-a177-8bdcc0bc0b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:42.056438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c51c89-aa4a-4fc6-a177-8bdcc0bc0b57 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:42.056746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b027ef-5ec7-46fc-b64c-d994aee49798 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:42.057100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c51c89-aa4a-4fc6-a177-8bdcc0bc0b57 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:42.057409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c51c89-aa4a-4fc6-a177-8bdcc0bc0b57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:42.058807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c51c89-aa4a-4fc6-a177-8bdcc0bc0b57 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:42.066779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 41/79] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:45:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:42.067580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c51c89-aa4a-4fc6-a177-8bdcc0bc0b57 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:42.093193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a113405-d6cf-4db5-aaeb-3992f33aa8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:42.093813 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a113405-d6cf-4db5-aaeb-3992f33aa8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:42.094103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a113405-d6cf-4db5-aaeb-3992f33aa8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:42.094383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a113405-d6cf-4db5-aaeb-3992f33aa8b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:42.095089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a113405-d6cf-4db5-aaeb-3992f33aa8b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:42.095399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a113405-d6cf-4db5-aaeb-3992f33aa8b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:42.096832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a113405-d6cf-4db5-aaeb-3992f33aa8b4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:42.104773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 39/80] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:45:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:42.105558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a113405-d6cf-4db5-aaeb-3992f33aa8b4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:42.154500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:42.172711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c51c89-aa4a-4fc6-a177-8bdcc0bc0b57 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:42.195574 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:42.204065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a113405-d6cf-4db5-aaeb-3992f33aa8b4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:42.205740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a358744-1b9f-4bd5-b5aa-dabd65fba740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:42.206373 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a358744-1b9f-4bd5-b5aa-dabd65fba740 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:42.206660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a358744-1b9f-4bd5-b5aa-dabd65fba740 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:42.206972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a358744-1b9f-4bd5-b5aa-dabd65fba740 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:42.208122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a358744-1b9f-4bd5-b5aa-dabd65fba740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:42.209581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a358744-1b9f-4bd5-b5aa-dabd65fba740 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:42.211489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 42/81] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:42.212018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a358744-1b9f-4bd5-b5aa-dabd65fba740 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:42.237888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0c2bd-2fcb-4675-b294-759e2bb33f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:42.238863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0c2bd-2fcb-4675-b294-759e2bb33f0c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:42.239153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0c2bd-2fcb-4675-b294-759e2bb33f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:42.239435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0c2bd-2fcb-4675-b294-759e2bb33f0c None admin] PATH_INFO: `/v3/users/4763f3bdfee74904a41057be46f076ed` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:42.240625 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0c2bd-2fcb-4675-b294-759e2bb33f0c None admin] RBAC: Authorizing `identity:get_user(user_id=4763f3bdfee74904a41057be46f076ed)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:42.242104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0c2bd-2fcb-4675-b294-759e2bb33f0c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:42.244011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 40/82] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:42 2022] GET /identity/v3/users/4763f3bdfee74904a41057be46f076ed => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:42.244794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0c2bd-2fcb-4675-b294-759e2bb33f0c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:42.313125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a358744-1b9f-4bd5-b5aa-dabd65fba740 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:42.345934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fc7b2-9104-40d4-a24b-db71d6a4105b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:42.346540 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fc7b2-9104-40d4-a24b-db71d6a4105b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:42.346637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0c2bd-2fcb-4675-b294-759e2bb33f0c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:42.346860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fc7b2-9104-40d4-a24b-db71d6a4105b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:42.347143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fc7b2-9104-40d4-a24b-db71d6a4105b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:42.365311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:42.379660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71a310-b610-45e1-bf1c-6d209395673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:42.380287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71a310-b610-45e1-bf1c-6d209395673a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:42.380571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71a310-b610-45e1-bf1c-6d209395673a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:42.382422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71a310-b610-45e1-bf1c-6d209395673a None admin] PATH_INFO: `/v3/users/71f957f6c8724676af13c99ddc4b62e3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:42.382422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71a310-b610-45e1-bf1c-6d209395673a None admin] RBAC: Authorizing `identity:get_user(user_id=71f957f6c8724676af13c99ddc4b62e3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:42.383479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71a310-b610-45e1-bf1c-6d209395673a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:42.385107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 41/83] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:42 2022] GET /identity/v3/users/71f957f6c8724676af13c99ddc4b62e3 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:42.385875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71a310-b610-45e1-bf1c-6d209395673a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:42.415241 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fc7b2-9104-40d4-a24b-db71d6a4105b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:42.416724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fc7b2-9104-40d4-a24b-db71d6a4105b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:42.417821 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:42.486439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71a310-b610-45e1-bf1c-6d209395673a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:42.515304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 43/84] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:42.520727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fc7b2-9104-40d4-a24b-db71d6a4105b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:42.521988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f28e9-1903-4516-aaf4-e7da64da36ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:42.522624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f28e9-1903-4516-aaf4-e7da64da36ab None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:42.522948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f28e9-1903-4516-aaf4-e7da64da36ab None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:42.523226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f28e9-1903-4516-aaf4-e7da64da36ab None admin] PATH_INFO: `/v3/users/71f957f6c8724676af13c99ddc4b62e3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:42.524395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f28e9-1903-4516-aaf4-e7da64da36ab None admin] RBAC: Authorizing `identity:get_user(user_id=71f957f6c8724676af13c99ddc4b62e3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:42.525848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f28e9-1903-4516-aaf4-e7da64da36ab None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:42.527469 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 42/85] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:42 2022] GET /identity/v3/users/71f957f6c8724676af13c99ddc4b62e3 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:42.528997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f28e9-1903-4516-aaf4-e7da64da36ab None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:42.569915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:42.619494 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fc7b2-9104-40d4-a24b-db71d6a4105b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:42.620957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:42.626444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f28e9-1903-4516-aaf4-e7da64da36ab None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:42.653034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa3343-6306-49b5-8474-a898950a6ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:42.654182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa3343-6306-49b5-8474-a898950a6ded None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:42.654467 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa3343-6306-49b5-8474-a898950a6ded None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:42.654778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa3343-6306-49b5-8474-a898950a6ded None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:42.655421 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa3343-6306-49b5-8474-a898950a6ded None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:42.656863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa3343-6306-49b5-8474-a898950a6ded None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:42.660085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9e342-86e5-4845-a8ab-eda82a7397a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:42.660686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9e342-86e5-4845-a8ab-eda82a7397a6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:42.660964 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9e342-86e5-4845-a8ab-eda82a7397a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:42.661236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9e342-86e5-4845-a8ab-eda82a7397a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:42.726224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9e342-86e5-4845-a8ab-eda82a7397a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:42.727730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9e342-86e5-4845-a8ab-eda82a7397a6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:42.738292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 44/86] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:45:42 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:45:42.739201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa3343-6306-49b5-8474-a898950a6ded None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:42.834861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:42.841996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 43/87] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:42.842911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9e342-86e5-4845-a8ab-eda82a7397a6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:42.867430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa3343-6306-49b5-8474-a898950a6ded None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:42.877464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:42.909376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515aac51-af3b-4933-9bc4-ba40b7cbf197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:42.910374 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515aac51-af3b-4933-9bc4-ba40b7cbf197 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:42.910481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515aac51-af3b-4933-9bc4-ba40b7cbf197 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:42.910736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515aac51-af3b-4933-9bc4-ba40b7cbf197 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:42.954015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9e342-86e5-4845-a8ab-eda82a7397a6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:42.988175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515aac51-af3b-4933-9bc4-ba40b7cbf197 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:42.989898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515aac51-af3b-4933-9bc4-ba40b7cbf197 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:42.999709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68468a73-bcb0-4717-8953-c7ab9fdc4b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:42.999709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68468a73-bcb0-4717-8953-c7ab9fdc4b63 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:42.999709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68468a73-bcb0-4717-8953-c7ab9fdc4b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:43.000052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68468a73-bcb0-4717-8953-c7ab9fdc4b63 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:43.002881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68468a73-bcb0-4717-8953-c7ab9fdc4b63 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:43.003487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68468a73-bcb0-4717-8953-c7ab9fdc4b63 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:43.005217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 44/88] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:42 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:43.005933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68468a73-bcb0-4717-8953-c7ab9fdc4b63 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:43.087447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:43.129086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 45/89] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:43.129981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515aac51-af3b-4933-9bc4-ba40b7cbf197 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:43.140136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68468a73-bcb0-4717-8953-c7ab9fdc4b63 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:43.143352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:43.184923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473005b6-1d01-45be-ae29-8d6dc2f1b265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:43.185554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473005b6-1d01-45be-ae29-8d6dc2f1b265 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:43.185842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473005b6-1d01-45be-ae29-8d6dc2f1b265 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:43.186122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473005b6-1d01-45be-ae29-8d6dc2f1b265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:43.260899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515aac51-af3b-4933-9bc4-ba40b7cbf197 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:43.272105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473005b6-1d01-45be-ae29-8d6dc2f1b265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:43.273700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473005b6-1d01-45be-ae29-8d6dc2f1b265 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:43.297891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0378ac9-6289-4237-aeec-a0978c9a0858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:43.298543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0378ac9-6289-4237-aeec-a0978c9a0858 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:43.298863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0378ac9-6289-4237-aeec-a0978c9a0858 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:43.299147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0378ac9-6289-4237-aeec-a0978c9a0858 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:43.300318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0378ac9-6289-4237-aeec-a0978c9a0858 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:43.301790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0378ac9-6289-4237-aeec-a0978c9a0858 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:43.303828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 46/90] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:43 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:43.304709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0378ac9-6289-4237-aeec-a0978c9a0858 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:43.341504 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:43.382848 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:43.388974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 45/91] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:43.390765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473005b6-1d01-45be-ae29-8d6dc2f1b265 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:43.409837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0378ac9-6289-4237-aeec-a0978c9a0858 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:43.445669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555603e-a1ea-46a4-b728-77b075d373a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:43.446313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555603e-a1ea-46a4-b728-77b075d373a9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:43.446605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555603e-a1ea-46a4-b728-77b075d373a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:43.446931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555603e-a1ea-46a4-b728-77b075d373a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:43.447678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555603e-a1ea-46a4-b728-77b075d373a9 None admin] RBAC: Adding query filter params (role.id=991609c9013b45b3907788ef9b42d051, scope.project.id=1231532f5f8e4acab98360e05e879050, user.id=71f957f6c8724676af13c99ddc4b62e3) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:43.447997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555603e-a1ea-46a4-b728-77b075d373a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:43.449460 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555603e-a1ea-46a4-b728-77b075d373a9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:43.462981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 47/92] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:43 2022] GET /identity/v3/role_assignments?user.id=71f957f6c8724676af13c99ddc4b62e3&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=991609c9013b45b3907788ef9b42d051 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:43.463959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555603e-a1ea-46a4-b728-77b075d373a9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:43.504938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473005b6-1d01-45be-ae29-8d6dc2f1b265 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:43.546818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44790e8f-e053-4491-96d8-16154f2cab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:43.547711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44790e8f-e053-4491-96d8-16154f2cab95 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:43.548181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44790e8f-e053-4491-96d8-16154f2cab95 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:43.548594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44790e8f-e053-4491-96d8-16154f2cab95 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:43.550013 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44790e8f-e053-4491-96d8-16154f2cab95 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:43.551686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44790e8f-e053-4491-96d8-16154f2cab95 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:43.553656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 46/93] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:43 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:43.554575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44790e8f-e053-4491-96d8-16154f2cab95 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:43.596227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:43.676476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44790e8f-e053-4491-96d8-16154f2cab95 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:43.681021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555603e-a1ea-46a4-b728-77b075d373a9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:43.724174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff36e3-0a1e-4ef2-859b-3d10adcbcf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:43.725185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff36e3-0a1e-4ef2-859b-3d10adcbcf12 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:43.725662 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff36e3-0a1e-4ef2-859b-3d10adcbcf12 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:43.726130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff36e3-0a1e-4ef2-859b-3d10adcbcf12 None admin] PATH_INFO: `/v3/projects/e720fc242af64ea6bc7da78e8e880990` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:43.727540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff36e3-0a1e-4ef2-859b-3d10adcbcf12 None admin] RBAC: Authorizing `identity:get_project(project_id=e720fc242af64ea6bc7da78e8e880990)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:43.731535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff36e3-0a1e-4ef2-859b-3d10adcbcf12 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:43.737248 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 47/94] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:43 2022] GET /identity/v3/projects/e720fc242af64ea6bc7da78e8e880990 => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:43.738349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff36e3-0a1e-4ef2-859b-3d10adcbcf12 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:43.758318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bffb1c-ffa9-4477-b26c-7d15628152d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:43.759634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bffb1c-ffa9-4477-b26c-7d15628152d7 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:43.760070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bffb1c-ffa9-4477-b26c-7d15628152d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:43.760497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bffb1c-ffa9-4477-b26c-7d15628152d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:43.761390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bffb1c-ffa9-4477-b26c-7d15628152d7 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=1231532f5f8e4acab98360e05e879050, user.id=4763f3bdfee74904a41057be46f076ed) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:43.761848 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bffb1c-ffa9-4477-b26c-7d15628152d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:43.764576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bffb1c-ffa9-4477-b26c-7d15628152d7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:43.773213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 48/95] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:43 2022] GET /identity/v3/role_assignments?user.id=4763f3bdfee74904a41057be46f076ed&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=62eb3e525077429aacc0ada63300802e => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:43.774463 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bffb1c-ffa9-4477-b26c-7d15628152d7 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:43.777989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:43.877079 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:43.882388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff36e3-0a1e-4ef2-859b-3d10adcbcf12 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:43.931767 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c27318-a473-483a-b5e0-35e65cb4c8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:43.932737 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c27318-a473-483a-b5e0-35e65cb4c8ac None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:43.934795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c27318-a473-483a-b5e0-35e65cb4c8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:43.938402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c27318-a473-483a-b5e0-35e65cb4c8ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:43.939690 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c27318-a473-483a-b5e0-35e65cb4c8ac None admin] RBAC: Adding query filter params (role.id=64e24908c05145b49cc926880ab769cf, scope.project.id=1231532f5f8e4acab98360e05e879050, user.id=71f957f6c8724676af13c99ddc4b62e3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:43.940237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c27318-a473-483a-b5e0-35e65cb4c8ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:43.947566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c27318-a473-483a-b5e0-35e65cb4c8ac None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:43.956970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bffb1c-ffa9-4477-b26c-7d15628152d7 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:43.957370 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 48/96] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:43 2022] GET /identity/v3/role_assignments?user.id=71f957f6c8724676af13c99ddc4b62e3&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=64e24908c05145b49cc926880ab769cf => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:44.001887 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db736ff-a10c-4310-9917-a86d6261830f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:44.002928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db736ff-a10c-4310-9917-a86d6261830f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:44.003409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db736ff-a10c-4310-9917-a86d6261830f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:44.003878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db736ff-a10c-4310-9917-a86d6261830f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:44.004812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db736ff-a10c-4310-9917-a86d6261830f None admin] RBAC: Adding query filter params (role.id=991609c9013b45b3907788ef9b42d051, scope.project.id=e720fc242af64ea6bc7da78e8e880990, user.id=71f957f6c8724676af13c99ddc4b62e3) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:44.005331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db736ff-a10c-4310-9917-a86d6261830f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:44.007074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db736ff-a10c-4310-9917-a86d6261830f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:44.014436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 49/97] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:43 2022] GET /identity/v3/role_assignments?user.id=71f957f6c8724676af13c99ddc4b62e3&scope.project.id=e720fc242af64ea6bc7da78e8e880990&role.id=991609c9013b45b3907788ef9b42d051 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:45.514960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb3d6-cb5b-4219-9af4-09b0dab122eb None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:45.515496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb3d6-cb5b-4219-9af4-09b0dab122eb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:45.515496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cb3d6-cb5b-4219-9af4-09b0dab122eb None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:45.516988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 49/98] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:45.522052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d67c2-2044-4f04-b0b2-db5d3ca6f7ca None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:45.522868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d67c2-2044-4f04-b0b2-db5d3ca6f7ca None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:45.523358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d67c2-2044-4f04-b0b2-db5d3ca6f7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:45.549701 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-253d67c2-2044-4f04-b0b2-db5d3ca6f7ca None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:45.551275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-253d67c2-2044-4f04-b0b2-db5d3ca6f7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:45.626731 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:45.640010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 50/99] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:45.963693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cb3d6-cb5b-4219-9af4-09b0dab122eb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:46.077359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ab67-0081-4c9d-8a0f-c6a0a253690b None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:46.077708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ab67-0081-4c9d-8a0f-c6a0a253690b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:46.077933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ab67-0081-4c9d-8a0f-c6a0a253690b None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:46.079634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 51/100] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:45:46.086826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cb3d6-cb5b-4219-9af4-09b0dab122eb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:46.088918 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaccf29-1060-4d64-af7a-ec10dd00a02a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:46.089211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaccf29-1060-4d64-af7a-ec10dd00a02a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:46.089500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaccf29-1060-4d64-af7a-ec10dd00a02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:46.123712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5eaccf29-1060-4d64-af7a-ec10dd00a02a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:46.124899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5eaccf29-1060-4d64-af7a-ec10dd00a02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:46.139922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c3e2f-e9df-4c22-b49d-ca285528e0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:46.140572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c3e2f-e9df-4c22-b49d-ca285528e0ea None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:46.140858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c3e2f-e9df-4c22-b49d-ca285528e0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:46.141139 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c3e2f-e9df-4c22-b49d-ca285528e0ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:46.147383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c3e2f-e9df-4c22-b49d-ca285528e0ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:46.148852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c3e2f-e9df-4c22-b49d-ca285528e0ea None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:46.150798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 50/101] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:46.157675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c3e2f-e9df-4c22-b49d-ca285528e0ea None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:46.227941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:46.277025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 52/102] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:45:46.354380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c3e2f-e9df-4c22-b49d-ca285528e0ea None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:46.400868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938e3f3-9fc4-4931-a2a8-0f0bfa0ca497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:46.401483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938e3f3-9fc4-4931-a2a8-0f0bfa0ca497 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:46.401762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938e3f3-9fc4-4931-a2a8-0f0bfa0ca497 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:46.402031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938e3f3-9fc4-4931-a2a8-0f0bfa0ca497 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:46.402707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938e3f3-9fc4-4931-a2a8-0f0bfa0ca497 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:46.404265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938e3f3-9fc4-4931-a2a8-0f0bfa0ca497 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:46.492329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d9e64-8864-4081-a47e-bb2cc6f9f3c0 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:46.492627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d9e64-8864-4081-a47e-bb2cc6f9f3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:46.492827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d9e64-8864-4081-a47e-bb2cc6f9f3c0 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:46.494305 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 53/103] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:45:46.510810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa67d7-4612-4f94-84ab-1ca246aa7256 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:46.510960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 51/104] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:45:46 2022] POST /identity/v3/projects => generated 302 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:45:46.511394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa67d7-4612-4f94-84ab-1ca246aa7256 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:46.511728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa67d7-4612-4f94-84ab-1ca246aa7256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:46.539368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-10fa67d7-4612-4f94-84ab-1ca246aa7256 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:46.540694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-10fa67d7-4612-4f94-84ab-1ca246aa7256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:46.628934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 54/105] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:45:46.834989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938e3f3-9fc4-4931-a2a8-0f0bfa0ca497 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:46.836190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b1109-ab82-4ba6-9d39-c6a0ae8b059f None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:46.836705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b1109-ab82-4ba6-9d39-c6a0ae8b059f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:46.837201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b1109-ab82-4ba6-9d39-c6a0ae8b059f None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:46.838783 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 55/106] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:46.844696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fbef6-94e2-4150-8c9e-c0a9f73329f4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:46.845263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fbef6-94e2-4150-8c9e-c0a9f73329f4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:46.845747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fbef6-94e2-4150-8c9e-c0a9f73329f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:46.882058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a27fbef6-94e2-4150-8c9e-c0a9f73329f4 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:46.883277 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a27fbef6-94e2-4150-8c9e-c0a9f73329f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:46.980514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:46.996435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:47.014378 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938e3f3-9fc4-4931-a2a8-0f0bfa0ca497 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:47.023897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 56/107] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:45:47.031969 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fbef6-94e2-4150-8c9e-c0a9f73329f4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:47.072639 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4670393-9017-4dad-80ba-95641f13917a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:47.073325 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670393-9017-4dad-80ba-95641f13917a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.074185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670393-9017-4dad-80ba-95641f13917a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.074185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670393-9017-4dad-80ba-95641f13917a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.094546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4670393-9017-4dad-80ba-95641f13917a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f4670393-9017-4dad-80ba-95641f13917a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:47.099727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:47.100781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:45:47.101761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:45:47.103534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 52/108] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:47.109138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4670393-9017-4dad-80ba-95641f13917a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:47.153564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fbef6-94e2-4150-8c9e-c0a9f73329f4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:47.196126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a8c4b-6a76-487b-8f74-dc459108f686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:47.196784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a8c4b-6a76-487b-8f74-dc459108f686 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.197067 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a8c4b-6a76-487b-8f74-dc459108f686 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.197340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a8c4b-6a76-487b-8f74-dc459108f686 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.220834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6a8c4b-6a76-487b-8f74-dc459108f686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-4e6a8c4b-6a76-487b-8f74-dc459108f686 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:47.225960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:47.226856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:47.227626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:47.232466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 57/109] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:47.233959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb95b6-5aaf-4099-af24-a48be4b18013 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.234242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb95b6-5aaf-4099-af24-a48be4b18013 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.234517 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb95b6-5aaf-4099-af24-a48be4b18013 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.235807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 58/110] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:47.240645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb95b6-5aaf-4099-af24-a48be4b18013 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:47.274808 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4670393-9017-4dad-80ba-95641f13917a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:47.325153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d9e6c-cc04-4c58-91dc-9cb5a8f6444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:47.325778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d9e6c-cc04-4c58-91dc-9cb5a8f6444f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.326062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d9e6c-cc04-4c58-91dc-9cb5a8f6444f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.326339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d9e6c-cc04-4c58-91dc-9cb5a8f6444f None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.327075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d9e6c-cc04-4c58-91dc-9cb5a8f6444f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:47.327345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d9e6c-cc04-4c58-91dc-9cb5a8f6444f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:47.328886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d9e6c-cc04-4c58-91dc-9cb5a8f6444f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:47.330099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:47.338459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 53/111] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:45:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:47.339907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9867b-fa1e-4059-864f-231968bd47dc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.342517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9867b-fa1e-4059-864f-231968bd47dc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.342517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9867b-fa1e-4059-864f-231968bd47dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.353579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:47.360239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9dd9867b-fa1e-4059-864f-231968bd47dc None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:47.361670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9dd9867b-fa1e-4059-864f-231968bd47dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:47.412359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb95b6-5aaf-4099-af24-a48be4b18013 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:47.448829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef3785-e43b-43dc-9eda-040afab563c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:47.449364 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef3785-e43b-43dc-9eda-040afab563c3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.449650 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef3785-e43b-43dc-9eda-040afab563c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.449955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef3785-e43b-43dc-9eda-040afab563c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.450629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef3785-e43b-43dc-9eda-040afab563c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:47.450967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef3785-e43b-43dc-9eda-040afab563c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:47.452358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef3785-e43b-43dc-9eda-040afab563c3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:47.460765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 59/112] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:45:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:47.466623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef3785-e43b-43dc-9eda-040afab563c3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:47.504090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 54/113] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:47.504974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9867b-fa1e-4059-864f-231968bd47dc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:47.573676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef3785-e43b-43dc-9eda-040afab563c3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:47.584385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:47.608966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a45697-c7d8-4f95-a0e4-4d0b09b5f756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:47.609536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45697-c7d8-4f95-a0e4-4d0b09b5f756 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.609820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45697-c7d8-4f95-a0e4-4d0b09b5f756 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.610096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45697-c7d8-4f95-a0e4-4d0b09b5f756 None admin] PATH_INFO: `/v3/users/9eb5eb07900040518d4edbf29e170992` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.626334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a45697-c7d8-4f95-a0e4-4d0b09b5f756 None admin] RBAC: Authorizing `identity:get_user(user_id=9eb5eb07900040518d4edbf29e170992)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:47.627855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a45697-c7d8-4f95-a0e4-4d0b09b5f756 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:47.629929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 60/114] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:47 2022] GET /identity/v3/users/9eb5eb07900040518d4edbf29e170992 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:47.630840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a45697-c7d8-4f95-a0e4-4d0b09b5f756 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:47.715981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9867b-fa1e-4059-864f-231968bd47dc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:47.737020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a45697-c7d8-4f95-a0e4-4d0b09b5f756 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:47.772242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5ad11-9693-4cad-9fe4-7c8ffbe393c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:47.772884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5ad11-9693-4cad-9fe4-7c8ffbe393c5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.773168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5ad11-9693-4cad-9fe4-7c8ffbe393c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.773445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5ad11-9693-4cad-9fe4-7c8ffbe393c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.776819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3589e-0356-4523-bbfa-d7a0c302d3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:47.777455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3589e-0356-4523-bbfa-d7a0c302d3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.777731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3589e-0356-4523-bbfa-d7a0c302d3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.778059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3589e-0356-4523-bbfa-d7a0c302d3b9 None admin] PATH_INFO: `/v3/users/71f957f6c8724676af13c99ddc4b62e3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.779451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3589e-0356-4523-bbfa-d7a0c302d3b9 None admin] RBAC: Authorizing `identity:get_user(user_id=71f957f6c8724676af13c99ddc4b62e3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:47.780926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3589e-0356-4523-bbfa-d7a0c302d3b9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:47.787515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 55/115] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:47 2022] GET /identity/v3/users/71f957f6c8724676af13c99ddc4b62e3 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:47.787515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3589e-0356-4523-bbfa-d7a0c302d3b9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:47.792204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5ad11-9693-4cad-9fe4-7c8ffbe393c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:47.794092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d0b5ad11-9693-4cad-9fe4-7c8ffbe393c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:47.798545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:47.799482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:47.800268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:47.802444 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 61/116] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:45:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:47.804002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3bca-c46c-4717-b314-cc76b84ddeb2 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.804476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3bca-c46c-4717-b314-cc76b84ddeb2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.804758 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3bca-c46c-4717-b314-cc76b84ddeb2 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.810712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 62/117] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:47.810712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3bca-c46c-4717-b314-cc76b84ddeb2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:47.839079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:47.911576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3589e-0356-4523-bbfa-d7a0c302d3b9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:47.922129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3bca-c46c-4717-b314-cc76b84ddeb2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:47.951776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17800e6-c4c7-48f1-9321-bdf22e161ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:47.952388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17800e6-c4c7-48f1-9321-bdf22e161ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.952675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17800e6-c4c7-48f1-9321-bdf22e161ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.952952 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17800e6-c4c7-48f1-9321-bdf22e161ee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.958934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808eb55-e4d6-44a6-b5a0-67c75b5a848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:47.959625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808eb55-e4d6-44a6-b5a0-67c75b5a848b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:47.959898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808eb55-e4d6-44a6-b5a0-67c75b5a848b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:47.960174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808eb55-e4d6-44a6-b5a0-67c75b5a848b None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:47.961445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808eb55-e4d6-44a6-b5a0-67c75b5a848b None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:47.963146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808eb55-e4d6-44a6-b5a0-67c75b5a848b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:47.969762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 63/118] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:47 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:47.969762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808eb55-e4d6-44a6-b5a0-67c75b5a848b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:48.025684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17800e6-c4c7-48f1-9321-bdf22e161ee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:48.027333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17800e6-c4c7-48f1-9321-bdf22e161ee6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:48.030150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:48.070060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:48.077161 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808eb55-e4d6-44a6-b5a0-67c75b5a848b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:48.115682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e10fe-c607-415b-9409-4dc51e4b56e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:48.116568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e10fe-c607-415b-9409-4dc51e4b56e4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:48.117034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e10fe-c607-415b-9409-4dc51e4b56e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:48.117311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e10fe-c607-415b-9409-4dc51e4b56e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:48.117975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e10fe-c607-415b-9409-4dc51e4b56e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:48.118282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e10fe-c607-415b-9409-4dc51e4b56e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:48.119705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e10fe-c607-415b-9409-4dc51e4b56e4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:48.132927 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 64/119] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:48.132927 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65898246-898f-441a-b912-2bf670907216 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:48.132927 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65898246-898f-441a-b912-2bf670907216 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:48.132927 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65898246-898f-441a-b912-2bf670907216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:48.140501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 56/120] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:48.145757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17800e6-c4c7-48f1-9321-bdf22e161ee6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:48.149592 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-65898246-898f-441a-b912-2bf670907216 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:48.150903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-65898246-898f-441a-b912-2bf670907216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:48.222983 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 65/121] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:48.230888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65898246-898f-441a-b912-2bf670907216 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:48.274344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17800e6-c4c7-48f1-9321-bdf22e161ee6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:48.277175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:48.321063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117e8d7-fcaf-4d79-b362-31c0270efb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:48.322052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117e8d7-fcaf-4d79-b362-31c0270efb1a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:48.322616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117e8d7-fcaf-4d79-b362-31c0270efb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:48.323359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117e8d7-fcaf-4d79-b362-31c0270efb1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:48.333220 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:48.337288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117e8d7-fcaf-4d79-b362-31c0270efb1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a117e8d7-fcaf-4d79-b362-31c0270efb1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:48.344666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:48.345518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:45:48.346482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 57/122] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:45:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:48.347339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117e8d7-fcaf-4d79-b362-31c0270efb1a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:48.353769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65898246-898f-441a-b912-2bf670907216 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:48.396859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ce65e-0274-428f-95e8-e1e584d17c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:48.397805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ce65e-0274-428f-95e8-e1e584d17c88 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:48.398261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ce65e-0274-428f-95e8-e1e584d17c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:48.398741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ce65e-0274-428f-95e8-e1e584d17c88 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050/users/71f957f6c8724676af13c99ddc4b62e3/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:48.401787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ce65e-0274-428f-95e8-e1e584d17c88 None admin] RBAC: Authorizing `identity:create_grant(project_id=1231532f5f8e4acab98360e05e879050, user_id=71f957f6c8724676af13c99ddc4b62e3, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:48.404256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ce65e-0274-428f-95e8-e1e584d17c88 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:48.453917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117e8d7-fcaf-4d79-b362-31c0270efb1a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:48.494792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 66/123] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:45:48 2022] PUT /identity/v3/projects/1231532f5f8e4acab98360e05e879050/users/71f957f6c8724676af13c99ddc4b62e3/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:45:48.494792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ce65e-0274-428f-95e8-e1e584d17c88 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:48.516583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8283071-ee3f-4111-a015-1a2548bf843b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:48.517261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8283071-ee3f-4111-a015-1a2548bf843b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:48.517544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8283071-ee3f-4111-a015-1a2548bf843b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:48.517818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8283071-ee3f-4111-a015-1a2548bf843b None admin] PATH_INFO: `/v3/users/4763f3bdfee74904a41057be46f076ed` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:48.520842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8283071-ee3f-4111-a015-1a2548bf843b None admin] RBAC: Authorizing `identity:get_user(user_id=4763f3bdfee74904a41057be46f076ed)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:48.524686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8283071-ee3f-4111-a015-1a2548bf843b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:48.524686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 58/124] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:48 2022] GET /identity/v3/users/4763f3bdfee74904a41057be46f076ed => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:48.526334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8283071-ee3f-4111-a015-1a2548bf843b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:48.583014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:48.642733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ce65e-0274-428f-95e8-e1e584d17c88 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:48.686204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8283071-ee3f-4111-a015-1a2548bf843b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:48.689181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea8bfa-5b24-4220-8370-85dd182c1110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:48.690097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea8bfa-5b24-4220-8370-85dd182c1110 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:48.690378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea8bfa-5b24-4220-8370-85dd182c1110 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:48.690656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea8bfa-5b24-4220-8370-85dd182c1110 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:48.693565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea8bfa-5b24-4220-8370-85dd182c1110 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:48.695073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea8bfa-5b24-4220-8370-85dd182c1110 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:48.696896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 67/125] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:48 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:48.697516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea8bfa-5b24-4220-8370-85dd182c1110 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:48.723143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:48.734029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651ac69-14e4-4a19-8769-a92e25038863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:48.736293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651ac69-14e4-4a19-8769-a92e25038863 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:48.736293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651ac69-14e4-4a19-8769-a92e25038863 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:48.736293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651ac69-14e4-4a19-8769-a92e25038863 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:48.736293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651ac69-14e4-4a19-8769-a92e25038863 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:48.736546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651ac69-14e4-4a19-8769-a92e25038863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:48.737818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651ac69-14e4-4a19-8769-a92e25038863 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:48.750832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 59/126] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:45:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:48.750832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651ac69-14e4-4a19-8769-a92e25038863 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:48.825434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea8bfa-5b24-4220-8370-85dd182c1110 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:48.862156 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ff2a6-33b9-4fc3-804d-982a8130047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:48.863278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ff2a6-33b9-4fc3-804d-982a8130047a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:48.863803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ff2a6-33b9-4fc3-804d-982a8130047a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:48.864309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ff2a6-33b9-4fc3-804d-982a8130047a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:48.879109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ff2a6-33b9-4fc3-804d-982a8130047a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-529ff2a6-33b9-4fc3-804d-982a8130047a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:48.883988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:48.884994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:48.885795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 68/127] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:48.886442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ff2a6-33b9-4fc3-804d-982a8130047a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:48.893009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:48.902016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651ac69-14e4-4a19-8769-a92e25038863 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:48.946500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eec73e-fd41-4314-8eb7-952dc91cb210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:48.947938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eec73e-fd41-4314-8eb7-952dc91cb210 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:48.948420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eec73e-fd41-4314-8eb7-952dc91cb210 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:48.948868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eec73e-fd41-4314-8eb7-952dc91cb210 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:48.951521 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eec73e-fd41-4314-8eb7-952dc91cb210 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:48.953642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eec73e-fd41-4314-8eb7-952dc91cb210 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:48.956588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 60/128] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:48 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:48.957834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eec73e-fd41-4314-8eb7-952dc91cb210 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:48.972364 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:48.999137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ff2a6-33b9-4fc3-804d-982a8130047a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:49.036900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fabbf0-a555-42e5-9a09-9557d0dde62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:49.037612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fabbf0-a555-42e5-9a09-9557d0dde62c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:49.037883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fabbf0-a555-42e5-9a09-9557d0dde62c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:49.038226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fabbf0-a555-42e5-9a09-9557d0dde62c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:49.039384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fabbf0-a555-42e5-9a09-9557d0dde62c None admin] RBAC: Adding query filter params (role.id=b6562af5bda44cc18b203749c9cc1ef3, scope.project.id=1231532f5f8e4acab98360e05e879050, user.id=9eb5eb07900040518d4edbf29e170992) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:49.039765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fabbf0-a555-42e5-9a09-9557d0dde62c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:49.041792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fabbf0-a555-42e5-9a09-9557d0dde62c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:49.048033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 69/129] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:48 2022] GET /identity/v3/role_assignments?user.id=9eb5eb07900040518d4edbf29e170992&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=b6562af5bda44cc18b203749c9cc1ef3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:49.048666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fabbf0-a555-42e5-9a09-9557d0dde62c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:49.110556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eec73e-fd41-4314-8eb7-952dc91cb210 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:49.175048 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57307726-0b8e-4971-9aa2-a6c1bde3d137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:49.175771 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57307726-0b8e-4971-9aa2-a6c1bde3d137 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:49.176058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57307726-0b8e-4971-9aa2-a6c1bde3d137 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:49.176336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57307726-0b8e-4971-9aa2-a6c1bde3d137 None admin] PATH_INFO: `/v3/users/71f957f6c8724676af13c99ddc4b62e3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:49.177588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57307726-0b8e-4971-9aa2-a6c1bde3d137 None admin] RBAC: Authorizing `identity:get_user(user_id=71f957f6c8724676af13c99ddc4b62e3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:49.179266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57307726-0b8e-4971-9aa2-a6c1bde3d137 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:49.181272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 61/130] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:48 2022] GET /identity/v3/users/71f957f6c8724676af13c99ddc4b62e3 => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:49.181857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57307726-0b8e-4971-9aa2-a6c1bde3d137 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:49.223171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fabbf0-a555-42e5-9a09-9557d0dde62c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:49.224045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:49.273149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c7bb4-7331-41a6-ad6d-8662582ddda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:49.274728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c7bb4-7331-41a6-ad6d-8662582ddda5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:49.275182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c7bb4-7331-41a6-ad6d-8662582ddda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:49.275605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c7bb4-7331-41a6-ad6d-8662582ddda5 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:49.276857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c7bb4-7331-41a6-ad6d-8662582ddda5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:49.277316 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c7bb4-7331-41a6-ad6d-8662582ddda5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:49.279310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c7bb4-7331-41a6-ad6d-8662582ddda5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:49.286163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:49.290835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 70/131] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:45:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:49.292182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8108a1-ae45-4047-a56a-76202f8d61b9 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:49.292532 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8108a1-ae45-4047-a56a-76202f8d61b9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:49.292874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8108a1-ae45-4047-a56a-76202f8d61b9 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:49.294800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 71/132] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:49.299121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8108a1-ae45-4047-a56a-76202f8d61b9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:49.314945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57307726-0b8e-4971-9aa2-a6c1bde3d137 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:49.356970 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7072d-e216-4425-8df3-2f51dab63dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:49.357786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7072d-e216-4425-8df3-2f51dab63dd5 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:49.358074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7072d-e216-4425-8df3-2f51dab63dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:49.358637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7072d-e216-4425-8df3-2f51dab63dd5 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050/users/4763f3bdfee74904a41057be46f076ed/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:49.360587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7072d-e216-4425-8df3-2f51dab63dd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1231532f5f8e4acab98360e05e879050, user_id=4763f3bdfee74904a41057be46f076ed, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:49.362472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7072d-e216-4425-8df3-2f51dab63dd5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:49.411039 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8108a1-ae45-4047-a56a-76202f8d61b9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:49.447837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea26a82-c93a-4a2e-87b7-d9b29b4fc2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:49.448480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea26a82-c93a-4a2e-87b7-d9b29b4fc2fc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:49.448765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea26a82-c93a-4a2e-87b7-d9b29b4fc2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:49.449042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea26a82-c93a-4a2e-87b7-d9b29b4fc2fc None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:49.450190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea26a82-c93a-4a2e-87b7-d9b29b4fc2fc None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:49.451689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea26a82-c93a-4a2e-87b7-d9b29b4fc2fc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:49.453363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 72/133] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:49 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:49.454208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea26a82-c93a-4a2e-87b7-d9b29b4fc2fc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:49.466340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 62/134] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:45:49 2022] PUT /identity/v3/projects/1231532f5f8e4acab98360e05e879050/users/4763f3bdfee74904a41057be46f076ed/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:45:49.468397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e5c0c-dcd0-4b9c-a662-e76097f862b6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:49.472440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e5c0c-dcd0-4b9c-a662-e76097f862b6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:49.472440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e5c0c-dcd0-4b9c-a662-e76097f862b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:49.495233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1a0e5c0c-dcd0-4b9c-a662-e76097f862b6 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:49.498221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1a0e5c0c-dcd0-4b9c-a662-e76097f862b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:49.587250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:49.612265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 63/135] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:49.614854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e5c0c-dcd0-4b9c-a662-e76097f862b6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:49.666084 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea26a82-c93a-4a2e-87b7-d9b29b4fc2fc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:49.667087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:49.721327 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe62350-f2fb-40a6-9c67-3b5e54501c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:49.721981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe62350-f2fb-40a6-9c67-3b5e54501c65 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:49.722259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe62350-f2fb-40a6-9c67-3b5e54501c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:49.722534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe62350-f2fb-40a6-9c67-3b5e54501c65 None admin] PATH_INFO: `/v3/users/71f957f6c8724676af13c99ddc4b62e3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:49.723768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe62350-f2fb-40a6-9c67-3b5e54501c65 None admin] RBAC: Authorizing `identity:get_user(user_id=71f957f6c8724676af13c99ddc4b62e3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:49.725560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe62350-f2fb-40a6-9c67-3b5e54501c65 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:49.727181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 73/136] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:49 2022] GET /identity/v3/users/71f957f6c8724676af13c99ddc4b62e3 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:49.733384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe62350-f2fb-40a6-9c67-3b5e54501c65 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:49.831459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e5c0c-dcd0-4b9c-a662-e76097f862b6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:49.900102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe62350-f2fb-40a6-9c67-3b5e54501c65 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:49.919758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431979d-22d7-46af-b801-feb54f5bcc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:49.920455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431979d-22d7-46af-b801-feb54f5bcc6b None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:49.920742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431979d-22d7-46af-b801-feb54f5bcc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:49.921024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431979d-22d7-46af-b801-feb54f5bcc6b None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050/users/71f957f6c8724676af13c99ddc4b62e3/roles/64e24908c05145b49cc926880ab769cf` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:49.928538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431979d-22d7-46af-b801-feb54f5bcc6b None admin] RBAC: Authorizing `identity:create_grant(project_id=1231532f5f8e4acab98360e05e879050, user_id=71f957f6c8724676af13c99ddc4b62e3, role_id=64e24908c05145b49cc926880ab769cf)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:49.930022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431979d-22d7-46af-b801-feb54f5bcc6b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:49.938271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34121c00-97aa-4c62-9e19-a7e8506f90a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:49.938956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34121c00-97aa-4c62-9e19-a7e8506f90a4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:49.939557 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34121c00-97aa-4c62-9e19-a7e8506f90a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:49.939837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34121c00-97aa-4c62-9e19-a7e8506f90a4 None admin] PATH_INFO: `/v3/projects/e720fc242af64ea6bc7da78e8e880990` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:49.940994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34121c00-97aa-4c62-9e19-a7e8506f90a4 None admin] RBAC: Authorizing `identity:get_project(project_id=e720fc242af64ea6bc7da78e8e880990)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:49.942506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34121c00-97aa-4c62-9e19-a7e8506f90a4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:49.944617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 74/137] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:49 2022] GET /identity/v3/projects/e720fc242af64ea6bc7da78e8e880990 => generated 312 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:49.951408 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34121c00-97aa-4c62-9e19-a7e8506f90a4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:49.959023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:50.010101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 64/138] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:45:49 2022] PUT /identity/v3/projects/1231532f5f8e4acab98360e05e879050/users/71f957f6c8724676af13c99ddc4b62e3/roles/64e24908c05145b49cc926880ab769cf => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:45:50.063613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431979d-22d7-46af-b801-feb54f5bcc6b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:50.119797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34121c00-97aa-4c62-9e19-a7e8506f90a4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:50.189410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebe11c-bbde-4c22-87da-0568e13d94e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:50.190103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebe11c-bbde-4c22-87da-0568e13d94e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:50.190384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebe11c-bbde-4c22-87da-0568e13d94e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:50.190659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebe11c-bbde-4c22-87da-0568e13d94e2 None admin] PATH_INFO: `/v3/projects/e720fc242af64ea6bc7da78e8e880990/users/71f957f6c8724676af13c99ddc4b62e3/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:50.194032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ebe11c-bbde-4c22-87da-0568e13d94e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e720fc242af64ea6bc7da78e8e880990, user_id=71f957f6c8724676af13c99ddc4b62e3, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:50.196431 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ebe11c-bbde-4c22-87da-0568e13d94e2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:50.227090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:50.232342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431979d-22d7-46af-b801-feb54f5bcc6b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:50.276049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 75/139] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:45:49 2022] PUT /identity/v3/projects/e720fc242af64ea6bc7da78e8e880990/users/71f957f6c8724676af13c99ddc4b62e3/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:45:50.289219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f33223-7906-4de3-aed0-4edbfac6a318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:50.289898 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f33223-7906-4de3-aed0-4edbfac6a318 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:50.290180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f33223-7906-4de3-aed0-4edbfac6a318 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:50.290458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f33223-7906-4de3-aed0-4edbfac6a318 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:50.291739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f33223-7906-4de3-aed0-4edbfac6a318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:50.294939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f33223-7906-4de3-aed0-4edbfac6a318 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:50.297161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 65/140] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:50.302425 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebe11c-bbde-4c22-87da-0568e13d94e2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:50.373697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:50.415374 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebe11c-bbde-4c22-87da-0568e13d94e2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:50.454027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4f47e-c6f5-4b91-8450-b613e3d6dd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:50.454583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4f47e-c6f5-4b91-8450-b613e3d6dd5e None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:50.454912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4f47e-c6f5-4b91-8450-b613e3d6dd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:50.455192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4f47e-c6f5-4b91-8450-b613e3d6dd5e None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:50.455883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4f47e-c6f5-4b91-8450-b613e3d6dd5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:50.457376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4f47e-c6f5-4b91-8450-b613e3d6dd5e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:50.545369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 76/141] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:45:50 2022] POST /identity/v3/users => generated 283 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:45:51.262786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d027f3-3fb5-45f0-b001-f8a3497b42cd None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:51.263293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d027f3-3fb5-45f0-b001-f8a3497b42cd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:51.263392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d027f3-3fb5-45f0-b001-f8a3497b42cd None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:51.265032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 66/142] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:51.270336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edddf6-ad68-41ef-8e10-948a3fb20158 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:51.270769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edddf6-ad68-41ef-8e10-948a3fb20158 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:51.270939 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edddf6-ad68-41ef-8e10-948a3fb20158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:51.294904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e1edddf6-ad68-41ef-8e10-948a3fb20158 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:51.296127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e1edddf6-ad68-41ef-8e10-948a3fb20158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:51.386052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 77/143] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:51.955609 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5d8a2-304a-4e05-bf58-f96594f25373 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:51.955954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5d8a2-304a-4e05-bf58-f96594f25373 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:51.956170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5d8a2-304a-4e05-bf58-f96594f25373 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:51.957685 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 67/144] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:51.960829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edddf6-ad68-41ef-8e10-948a3fb20158 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:52.004695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d21b32-77f5-4a48-b580-e2747fe09a64 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:52.004877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d21b32-77f5-4a48-b580-e2747fe09a64 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:52.005158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d21b32-77f5-4a48-b580-e2747fe09a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:52.038799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-94d21b32-77f5-4a48-b580-e2747fe09a64 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:52.039351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:52.040172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-94d21b32-77f5-4a48-b580-e2747fe09a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:52.101499 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edddf6-ad68-41ef-8e10-948a3fb20158 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:52.155122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc745592-8c91-40e4-8bf8-df489865fe66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:52.156472 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc745592-8c91-40e4-8bf8-df489865fe66 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:52.156991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc745592-8c91-40e4-8bf8-df489865fe66 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:52.157473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc745592-8c91-40e4-8bf8-df489865fe66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:52.179273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc745592-8c91-40e4-8bf8-df489865fe66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:52.180339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 68/145] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-dc745592-8c91-40e4-8bf8-df489865fe66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:52.185508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:52.186370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:52.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:52.189671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 78/146] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:52.195530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d21b32-77f5-4a48-b580-e2747fe09a64 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:52.308091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:52.336108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d21b32-77f5-4a48-b580-e2747fe09a64 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:52.382815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd4ca7-9d94-4a8d-aaca-5395634de163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:52.383431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd4ca7-9d94-4a8d-aaca-5395634de163 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:52.383708 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd4ca7-9d94-4a8d-aaca-5395634de163 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:52.383976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd4ca7-9d94-4a8d-aaca-5395634de163 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:52.384646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd4ca7-9d94-4a8d-aaca-5395634de163 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:52.384947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd4ca7-9d94-4a8d-aaca-5395634de163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:52.386320 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd4ca7-9d94-4a8d-aaca-5395634de163 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:52.395064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 69/147] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:45:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:52.401064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc745592-8c91-40e4-8bf8-df489865fe66 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:52.556484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc745592-8c91-40e4-8bf8-df489865fe66 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:52.572061 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:52.598143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdcafc-4b3c-459d-b59b-1071768a39ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:52.599243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdcafc-4b3c-459d-b59b-1071768a39ef None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:52.600268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdcafc-4b3c-459d-b59b-1071768a39ef None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:52.600268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdcafc-4b3c-459d-b59b-1071768a39ef None admin] PATH_INFO: `/v3/users/71f957f6c8724676af13c99ddc4b62e3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:52.603435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdcafc-4b3c-459d-b59b-1071768a39ef None admin] RBAC: Authorizing `identity:get_user(user_id=71f957f6c8724676af13c99ddc4b62e3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:52.605142 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdcafc-4b3c-459d-b59b-1071768a39ef None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:52.607432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 79/148] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:52 2022] GET /identity/v3/users/71f957f6c8724676af13c99ddc4b62e3 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:52.613369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd4ca7-9d94-4a8d-aaca-5395634de163 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:52.673417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdcafc-4b3c-459d-b59b-1071768a39ef None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:52.814059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd4ca7-9d94-4a8d-aaca-5395634de163 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:52.870599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbef06-7735-40f1-b16e-d826b9ee6ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:52.871183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbef06-7735-40f1-b16e-d826b9ee6ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:52.871994 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbef06-7735-40f1-b16e-d826b9ee6ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:52.872273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbef06-7735-40f1-b16e-d826b9ee6ea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:52.899033 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:52.947415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdcafc-4b3c-459d-b59b-1071768a39ef None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:52.986348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbef06-7735-40f1-b16e-d826b9ee6ea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:52.987920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbef06-7735-40f1-b16e-d826b9ee6ea7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:53.025464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c47b06-f893-476e-bbdc-9f4de460d73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:53.026127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c47b06-f893-476e-bbdc-9f4de460d73a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:53.026409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c47b06-f893-476e-bbdc-9f4de460d73a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:53.026702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c47b06-f893-476e-bbdc-9f4de460d73a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:53.053770 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c47b06-f893-476e-bbdc-9f4de460d73a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b0c47b06-f893-476e-bbdc-9f4de460d73a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:53.059775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:53.060633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:45:53.061396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:53.067034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 80/149] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:45:53.077283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4aabb-8b06-44a9-8bd4-06d08415da15 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:53.077568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4aabb-8b06-44a9-8bd4-06d08415da15 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:53.077843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4aabb-8b06-44a9-8bd4-06d08415da15 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:53.079221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 81/150] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:53 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:53.092352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d26eb-7f73-471c-953d-75736ee94300 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:53.092635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d26eb-7f73-471c-953d-75736ee94300 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:53.092898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d26eb-7f73-471c-953d-75736ee94300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:53.113129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-bb5d26eb-7f73-471c-953d-75736ee94300 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:53.114343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bb5d26eb-7f73-471c-953d-75736ee94300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:53.169288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:53.200814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 82/151] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:45:53.207945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d26eb-7f73-471c-953d-75736ee94300 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:53.242522 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 70/152] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:53.250907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbef06-7735-40f1-b16e-d826b9ee6ea7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:53.342430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:53.414946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d26eb-7f73-471c-953d-75736ee94300 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:53.452316 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbef06-7735-40f1-b16e-d826b9ee6ea7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:53.473419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bd730-69b4-499b-891b-fbcc3de6f42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:53.474485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd730-69b4-499b-891b-fbcc3de6f42c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:53.474981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd730-69b4-499b-891b-fbcc3de6f42c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:53.475536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bd730-69b4-499b-891b-fbcc3de6f42c None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:53.476398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bd730-69b4-499b-891b-fbcc3de6f42c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:53.476897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bd730-69b4-499b-891b-fbcc3de6f42c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:53.478582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bd730-69b4-499b-891b-fbcc3de6f42c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:53.494539 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 83/153] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:45:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:53.496142 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9b967-72d5-45a1-bc1f-201cc54769d1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:53.497205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9b967-72d5-45a1-bc1f-201cc54769d1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:53.497678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9b967-72d5-45a1-bc1f-201cc54769d1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:53.499158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 84/154] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:53.502628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9b967-72d5-45a1-bc1f-201cc54769d1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:53.520778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada1aef-a759-4ca0-a949-dae3ab85eb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:53.521426 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada1aef-a759-4ca0-a949-dae3ab85eb59 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:53.521703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada1aef-a759-4ca0-a949-dae3ab85eb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:53.521971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada1aef-a759-4ca0-a949-dae3ab85eb59 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:53.527386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada1aef-a759-4ca0-a949-dae3ab85eb59 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:53.528920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada1aef-a759-4ca0-a949-dae3ab85eb59 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:53.530915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 71/155] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:53 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:53.532067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668f1e-1ee2-4cf5-84dd-4ac7a31e0639 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:53.532346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668f1e-1ee2-4cf5-84dd-4ac7a31e0639 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:53.532620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668f1e-1ee2-4cf5-84dd-4ac7a31e0639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:53.566236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dd668f1e-1ee2-4cf5-84dd-4ac7a31e0639 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:53.567511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dd668f1e-1ee2-4cf5-84dd-4ac7a31e0639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:53.579023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:53.693085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9b967-72d5-45a1-bc1f-201cc54769d1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:53.725145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 72/156] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:53.725960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd668f1e-1ee2-4cf5-84dd-4ac7a31e0639 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:53.731923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330d995-acec-47f6-b389-f4f470bb3993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:53.732726 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330d995-acec-47f6-b389-f4f470bb3993 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:53.733139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330d995-acec-47f6-b389-f4f470bb3993 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:53.733513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330d995-acec-47f6-b389-f4f470bb3993 None admin] PATH_INFO: `/v3/users/9eb5eb07900040518d4edbf29e170992` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:53.735087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330d995-acec-47f6-b389-f4f470bb3993 None admin] RBAC: Authorizing `identity:get_user(user_id=9eb5eb07900040518d4edbf29e170992)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:53.740522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330d995-acec-47f6-b389-f4f470bb3993 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:53.742883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 85/157] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:53 2022] GET /identity/v3/users/9eb5eb07900040518d4edbf29e170992 => generated 289 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:53.743977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330d995-acec-47f6-b389-f4f470bb3993 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:53.841179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:53.851587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:53.855353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330d995-acec-47f6-b389-f4f470bb3993 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:53.953048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8226a51-73b4-4be2-a669-a1299d61e819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:53.953693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8226a51-73b4-4be2-a669-a1299d61e819 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:53.953975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8226a51-73b4-4be2-a669-a1299d61e819 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:53.954247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8226a51-73b4-4be2-a669-a1299d61e819 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:53.982350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8226a51-73b4-4be2-a669-a1299d61e819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-c8226a51-73b4-4be2-a669-a1299d61e819 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:53.987214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:53.988120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:53.988904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 86/158] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:45:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:53.992655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91880b0e-fdd8-4e81-b097-eb731f424cf7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:53.992955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91880b0e-fdd8-4e81-b097-eb731f424cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:53.993244 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91880b0e-fdd8-4e81-b097-eb731f424cf7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:53.994532 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 87/159] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:53.995285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91880b0e-fdd8-4e81-b097-eb731f424cf7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:53.999954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd668f1e-1ee2-4cf5-84dd-4ac7a31e0639 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:54.057260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d6c13-8d2d-4495-bd64-5eff30d56ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:54.057880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d6c13-8d2d-4495-bd64-5eff30d56ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:54.058160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d6c13-8d2d-4495-bd64-5eff30d56ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:54.058432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d6c13-8d2d-4495-bd64-5eff30d56ec8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:54.059768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d6c13-8d2d-4495-bd64-5eff30d56ec8 None admin] RBAC: Adding query filter params (role.id=991609c9013b45b3907788ef9b42d051, scope.project.id=1231532f5f8e4acab98360e05e879050, user.id=71f957f6c8724676af13c99ddc4b62e3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:54.060079 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d6c13-8d2d-4495-bd64-5eff30d56ec8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:54.061551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d6c13-8d2d-4495-bd64-5eff30d56ec8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:54.073168 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 73/160] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:53 2022] GET /identity/v3/role_assignments?user.id=71f957f6c8724676af13c99ddc4b62e3&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=991609c9013b45b3907788ef9b42d051 => generated 608 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:54.074217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d6c13-8d2d-4495-bd64-5eff30d56ec8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:54.202154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91880b0e-fdd8-4e81-b097-eb731f424cf7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:54.268894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29333e-8dd0-4ef5-8789-eefcfde4a56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:54.269463 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29333e-8dd0-4ef5-8789-eefcfde4a56e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:54.269743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29333e-8dd0-4ef5-8789-eefcfde4a56e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:54.270019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29333e-8dd0-4ef5-8789-eefcfde4a56e None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:54.271312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29333e-8dd0-4ef5-8789-eefcfde4a56e None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:54.272782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29333e-8dd0-4ef5-8789-eefcfde4a56e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:54.274481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 88/161] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:53 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:54.275690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa8a7d-8952-40a8-bfa1-420e35ace707 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:54.275975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa8a7d-8952-40a8-bfa1-420e35ace707 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:54.276248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa8a7d-8952-40a8-bfa1-420e35ace707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:54.280684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d6c13-8d2d-4495-bd64-5eff30d56ec8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:54.282491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:54.318721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-88aa8a7d-8952-40a8-bfa1-420e35ace707 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:54.320343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-88aa8a7d-8952-40a8-bfa1-420e35ace707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:54.326870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedcd2d-aab3-4477-9e23-3ad997f15cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:54.328463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedcd2d-aab3-4477-9e23-3ad997f15cef None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:54.328768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedcd2d-aab3-4477-9e23-3ad997f15cef None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:54.329050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedcd2d-aab3-4477-9e23-3ad997f15cef None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:54.329740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedcd2d-aab3-4477-9e23-3ad997f15cef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:54.330048 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedcd2d-aab3-4477-9e23-3ad997f15cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:54.331493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fedcd2d-aab3-4477-9e23-3ad997f15cef None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:54.348680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 74/162] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:54.349415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedcd2d-aab3-4477-9e23-3ad997f15cef None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:54.360832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:54.464190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 89/163] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:54.464968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa8a7d-8952-40a8-bfa1-420e35ace707 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:54.577741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa8a7d-8952-40a8-bfa1-420e35ace707 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:54.598426 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedcd2d-aab3-4477-9e23-3ad997f15cef None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:54.615734 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ebb7c-ada1-4c7c-b5a0-4921853d2ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:54.616630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ebb7c-ada1-4c7c-b5a0-4921853d2ee1 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:54.617058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ebb7c-ada1-4c7c-b5a0-4921853d2ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:54.617528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ebb7c-ada1-4c7c-b5a0-4921853d2ee1 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050/users/9eb5eb07900040518d4edbf29e170992/roles/b6562af5bda44cc18b203749c9cc1ef3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:54.619715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ebb7c-ada1-4c7c-b5a0-4921853d2ee1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1231532f5f8e4acab98360e05e879050, user_id=9eb5eb07900040518d4edbf29e170992, role_id=b6562af5bda44cc18b203749c9cc1ef3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:54.621361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ebb7c-ada1-4c7c-b5a0-4921853d2ee1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:54.653548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954d93-9b2f-400c-89a1-6c9ae4132bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:54.654609 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954d93-9b2f-400c-89a1-6c9ae4132bcb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:54.656968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954d93-9b2f-400c-89a1-6c9ae4132bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:54.657478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24954d93-9b2f-400c-89a1-6c9ae4132bcb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:54.670471 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24954d93-9b2f-400c-89a1-6c9ae4132bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-24954d93-9b2f-400c-89a1-6c9ae4132bcb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:54.680262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:54.681394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:45:54.682366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:45:54.684920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 75/164] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:54.686013 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954d93-9b2f-400c-89a1-6c9ae4132bcb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:54.709329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 90/165] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:45:54 2022] PUT /identity/v3/projects/1231532f5f8e4acab98360e05e879050/users/9eb5eb07900040518d4edbf29e170992/roles/b6562af5bda44cc18b203749c9cc1ef3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:45:54.711443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ebb7c-ada1-4c7c-b5a0-4921853d2ee1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:54.744438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:54.748258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:54.887670 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ebb7c-ada1-4c7c-b5a0-4921853d2ee1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:54.907867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24954d93-9b2f-400c-89a1-6c9ae4132bcb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:54.944206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a0689-cf9d-416f-b05d-b1267068d124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:54.944921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0689-cf9d-416f-b05d-b1267068d124 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:54.945203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0689-cf9d-416f-b05d-b1267068d124 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:54.945546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0689-cf9d-416f-b05d-b1267068d124 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:54.946422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a0689-cf9d-416f-b05d-b1267068d124 None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:54.946792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a0689-cf9d-416f-b05d-b1267068d124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:54.948210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a0689-cf9d-416f-b05d-b1267068d124 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:54.958614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 91/166] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:45:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:54.964702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a0689-cf9d-416f-b05d-b1267068d124 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:54.969212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a05172-2c33-4cd2-8813-25f4d78c1068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:54.970088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a05172-2c33-4cd2-8813-25f4d78c1068 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:54.970541 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a05172-2c33-4cd2-8813-25f4d78c1068 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:54.971111 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a05172-2c33-4cd2-8813-25f4d78c1068 None admin] PATH_INFO: `/v3/users/4763f3bdfee74904a41057be46f076ed` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:54.974753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a05172-2c33-4cd2-8813-25f4d78c1068 None admin] RBAC: Authorizing `identity:get_user(user_id=4763f3bdfee74904a41057be46f076ed)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:54.976464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a05172-2c33-4cd2-8813-25f4d78c1068 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:54.978310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 76/167] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:54 2022] GET /identity/v3/users/4763f3bdfee74904a41057be46f076ed => generated 247 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:54.979293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a05172-2c33-4cd2-8813-25f4d78c1068 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:55.081516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:55.091056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:55.102560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a0689-cf9d-416f-b05d-b1267068d124 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:55.104901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a05172-2c33-4cd2-8813-25f4d78c1068 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:55.157757 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239db1ef-90ac-4e20-9c38-ef2f97ed29ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:55.158389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239db1ef-90ac-4e20-9c38-ef2f97ed29ea None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:55.158696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239db1ef-90ac-4e20-9c38-ef2f97ed29ea None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:55.158970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239db1ef-90ac-4e20-9c38-ef2f97ed29ea None admin] PATH_INFO: `/v3/users/71f957f6c8724676af13c99ddc4b62e3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:55.160298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239db1ef-90ac-4e20-9c38-ef2f97ed29ea None admin] RBAC: Authorizing `identity:get_user(user_id=71f957f6c8724676af13c99ddc4b62e3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:55.167771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239db1ef-90ac-4e20-9c38-ef2f97ed29ea None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:55.169761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 92/168] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:54 2022] GET /identity/v3/users/71f957f6c8724676af13c99ddc4b62e3 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:45:55.170931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239db1ef-90ac-4e20-9c38-ef2f97ed29ea None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:55.192775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce4c61-e636-425e-8093-36dfefefc3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:55.193696 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce4c61-e636-425e-8093-36dfefefc3ab None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:55.194125 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce4c61-e636-425e-8093-36dfefefc3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:55.194566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce4c61-e636-425e-8093-36dfefefc3ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:55.229758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce4c61-e636-425e-8093-36dfefefc3ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-03ce4c61-e636-425e-8093-36dfefefc3ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:55.234709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:55.235581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:45:55.236373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 77/169] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:45:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:55.237024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce4c61-e636-425e-8093-36dfefefc3ab None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:55.366525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239db1ef-90ac-4e20-9c38-ef2f97ed29ea None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:55.431998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce4c61-e636-425e-8093-36dfefefc3ab None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:55.443722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d0b99-b36c-4c26-974d-4f1fec648536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:55.445241 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d0b99-b36c-4c26-974d-4f1fec648536 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:55.445241 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d0b99-b36c-4c26-974d-4f1fec648536 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:55.445241 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d0b99-b36c-4c26-974d-4f1fec648536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:55.483552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:55.509846 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566b13-4c4b-4274-945c-4ae2c26144a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:55.510473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566b13-4c4b-4274-945c-4ae2c26144a0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:55.510785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566b13-4c4b-4274-945c-4ae2c26144a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:55.511062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566b13-4c4b-4274-945c-4ae2c26144a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:55.579084 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:55.607686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d0b99-b36c-4c26-974d-4f1fec648536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:55.609242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d0b99-b36c-4c26-974d-4f1fec648536 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:55.622266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3566b13-4c4b-4274-945c-4ae2c26144a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:55.623812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3566b13-4c4b-4274-945c-4ae2c26144a0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:55.740203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 78/170] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:55.741328 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566b13-4c4b-4274-945c-4ae2c26144a0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:55.770992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 93/171] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:55.772521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c95de1-b381-4b14-8429-e7d97cc96bb7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:55.772707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c95de1-b381-4b14-8429-e7d97cc96bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:55.772983 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c95de1-b381-4b14-8429-e7d97cc96bb7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:55.775239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 94/172] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:55.776785 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4429ad-a16b-4f80-aab4-5b4b87330821 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:55.777297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4429ad-a16b-4f80-aab4-5b4b87330821 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:55.778110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4429ad-a16b-4f80-aab4-5b4b87330821 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:55.780099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 95/173] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:55.783277 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4429ad-a16b-4f80-aab4-5b4b87330821 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:55.809145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:55.846780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:55.862757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566b13-4c4b-4274-945c-4ae2c26144a0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:55.898471 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa79e9d-de3b-4f1a-be20-3e37f2cb9d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:55.899119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa79e9d-de3b-4f1a-be20-3e37f2cb9d7b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:55.899397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa79e9d-de3b-4f1a-be20-3e37f2cb9d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:55.899672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa79e9d-de3b-4f1a-be20-3e37f2cb9d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:55.900379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa79e9d-de3b-4f1a-be20-3e37f2cb9d7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:55.900695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa79e9d-de3b-4f1a-be20-3e37f2cb9d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:55.902084 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa79e9d-de3b-4f1a-be20-3e37f2cb9d7b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:55.904326 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4429ad-a16b-4f80-aab4-5b4b87330821 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:55.915591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 79/174] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:45:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:55.915591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa79e9d-de3b-4f1a-be20-3e37f2cb9d7b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:55.944674 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5637d95-17b4-4f86-a0fa-47fad1095324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:55.945307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5637d95-17b4-4f86-a0fa-47fad1095324 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:55.945589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5637d95-17b4-4f86-a0fa-47fad1095324 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:55.945856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5637d95-17b4-4f86-a0fa-47fad1095324 None admin] PATH_INFO: `/v3/projects/e720fc242af64ea6bc7da78e8e880990` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:55.947019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5637d95-17b4-4f86-a0fa-47fad1095324 None admin] RBAC: Authorizing `identity:get_project(project_id=e720fc242af64ea6bc7da78e8e880990)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:55.948428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5637d95-17b4-4f86-a0fa-47fad1095324 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:55.950269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 96/175] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:55 2022] GET /identity/v3/projects/e720fc242af64ea6bc7da78e8e880990 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:55.953130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb44d8-cd55-45af-af0f-8fb453fd78fc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:55.953650 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb44d8-cd55-45af-af0f-8fb453fd78fc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:55.954146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb44d8-cd55-45af-af0f-8fb453fd78fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:55.976974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-bdeb44d8-cd55-45af-af0f-8fb453fd78fc None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:55.978984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bdeb44d8-cd55-45af-af0f-8fb453fd78fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:56.020147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa79e9d-de3b-4f1a-be20-3e37f2cb9d7b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:56.036150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:56.051372 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 97/176] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:56.052902 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246ed8a-beea-4093-876d-871060c12dfb None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:56.053191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246ed8a-beea-4093-876d-871060c12dfb None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:56.053466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246ed8a-beea-4093-876d-871060c12dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:56.069344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c92fb4-f88c-49f6-b87b-97bd01ae9b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:56.069992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c92fb4-f88c-49f6-b87b-97bd01ae9b63 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:56.070276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c92fb4-f88c-49f6-b87b-97bd01ae9b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:56.070556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c92fb4-f88c-49f6-b87b-97bd01ae9b63 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:56.071733 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c92fb4-f88c-49f6-b87b-97bd01ae9b63 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:56.073214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c92fb4-f88c-49f6-b87b-97bd01ae9b63 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:56.081456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 80/177] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:55 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:56.081456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498f926-f691-44a5-b0f2-febd5bf8c770 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:56.081456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498f926-f691-44a5-b0f2-febd5bf8c770 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:56.081456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498f926-f691-44a5-b0f2-febd5bf8c770 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:56.081456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 81/178] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:56.081456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498f926-f691-44a5-b0f2-febd5bf8c770 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:56.091035 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1246ed8a-beea-4093-876d-871060c12dfb None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:56.092467 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1246ed8a-beea-4093-876d-871060c12dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:56.144103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:56.172302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 98/179] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:56.179733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246ed8a-beea-4093-876d-871060c12dfb None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:56.215951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498f926-f691-44a5-b0f2-febd5bf8c770 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:56.256431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428d5ed-0876-4723-a8fe-ce7a86d9e024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:56.257341 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428d5ed-0876-4723-a8fe-ce7a86d9e024 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:56.257769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428d5ed-0876-4723-a8fe-ce7a86d9e024 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:56.258189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428d5ed-0876-4723-a8fe-ce7a86d9e024 None admin] PATH_INFO: `/v3/users/71f957f6c8724676af13c99ddc4b62e3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:56.260116 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428d5ed-0876-4723-a8fe-ce7a86d9e024 None admin] RBAC: Authorizing `identity:get_user(user_id=71f957f6c8724676af13c99ddc4b62e3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:56.262313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428d5ed-0876-4723-a8fe-ce7a86d9e024 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:56.265105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 82/180] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:56 2022] GET /identity/v3/users/71f957f6c8724676af13c99ddc4b62e3 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:56.266148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428d5ed-0876-4723-a8fe-ce7a86d9e024 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:56.327337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:56.357147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246ed8a-beea-4093-876d-871060c12dfb None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:56.395231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428d5ed-0876-4723-a8fe-ce7a86d9e024 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:56.400728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0777084-215b-4362-91ec-78a4a2a3a538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:56.401463 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0777084-215b-4362-91ec-78a4a2a3a538 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:56.401837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0777084-215b-4362-91ec-78a4a2a3a538 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:56.402201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0777084-215b-4362-91ec-78a4a2a3a538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:56.403700 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0777084-215b-4362-91ec-78a4a2a3a538 None admin] RBAC: Adding query filter params (role.id=991609c9013b45b3907788ef9b42d051, scope.project.id=e720fc242af64ea6bc7da78e8e880990, user.id=71f957f6c8724676af13c99ddc4b62e3) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:56.404108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0777084-215b-4362-91ec-78a4a2a3a538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:56.406128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0777084-215b-4362-91ec-78a4a2a3a538 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:56.413731 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 99/181] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:56 2022] GET /identity/v3/role_assignments?user.id=71f957f6c8724676af13c99ddc4b62e3&scope.project.id=e720fc242af64ea6bc7da78e8e880990&role.id=991609c9013b45b3907788ef9b42d051 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:56.414753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161b2ca-2e31-4282-b950-8be2512b3fda None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:56.415123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161b2ca-2e31-4282-b950-8be2512b3fda None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:56.415492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161b2ca-2e31-4282-b950-8be2512b3fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:56.436797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0161b2ca-2e31-4282-b950-8be2512b3fda None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:56.438228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0161b2ca-2e31-4282-b950-8be2512b3fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:56.439488 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6121a2-fa48-4e8a-9f9b-ebb08620b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:56.440654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6121a2-fa48-4e8a-9f9b-ebb08620b1eb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:56.441434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6121a2-fa48-4e8a-9f9b-ebb08620b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:56.441860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6121a2-fa48-4e8a-9f9b-ebb08620b1eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:56.443598 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6121a2-fa48-4e8a-9f9b-ebb08620b1eb None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=1231532f5f8e4acab98360e05e879050, user.id=4763f3bdfee74904a41057be46f076ed) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:56.443918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6121a2-fa48-4e8a-9f9b-ebb08620b1eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:56.445446 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6121a2-fa48-4e8a-9f9b-ebb08620b1eb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:56.453171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 83/182] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:56 2022] GET /identity/v3/role_assignments?user.id=4763f3bdfee74904a41057be46f076ed&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=62eb3e525077429aacc0ada63300802e => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:56.454030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6121a2-fa48-4e8a-9f9b-ebb08620b1eb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:56.482956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:56.533741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 100/183] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:56.535585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161b2ca-2e31-4282-b950-8be2512b3fda None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:56.660144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161b2ca-2e31-4282-b950-8be2512b3fda None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:56.683247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6121a2-fa48-4e8a-9f9b-ebb08620b1eb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:56.702185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf2444-f869-4124-8680-50466558bfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:56.703481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf2444-f869-4124-8680-50466558bfda None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:56.703976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf2444-f869-4124-8680-50466558bfda None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:56.704451 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf2444-f869-4124-8680-50466558bfda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:56.708483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:56.719367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf2444-f869-4124-8680-50466558bfda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-0acf2444-f869-4124-8680-50466558bfda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:56.725681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:56.726617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:45:56.727476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:56.730044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 101/184] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:45:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:56.731527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf2444-f869-4124-8680-50466558bfda None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:56.760091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b8214-dc3c-4d1e-a728-d00e746e8466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:56.763503 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b8214-dc3c-4d1e-a728-d00e746e8466 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:56.763797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b8214-dc3c-4d1e-a728-d00e746e8466 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:56.764038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b8214-dc3c-4d1e-a728-d00e746e8466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:56.774874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:56.864006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf2444-f869-4124-8680-50466558bfda None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:56.887688 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b8214-dc3c-4d1e-a728-d00e746e8466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:56.889171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b8214-dc3c-4d1e-a728-d00e746e8466 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:56.907762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75d963-71aa-4353-b6e3-bed53efdd9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:56.909059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75d963-71aa-4353-b6e3-bed53efdd9fb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:56.909568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75d963-71aa-4353-b6e3-bed53efdd9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:56.910496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75d963-71aa-4353-b6e3-bed53efdd9fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:56.911896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75d963-71aa-4353-b6e3-bed53efdd9fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:56.914170 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75d963-71aa-4353-b6e3-bed53efdd9fb None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:56.916414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 102/185] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:45:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:56.917512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75d963-71aa-4353-b6e3-bed53efdd9fb None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:57.001802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 84/186] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:57.002643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b8214-dc3c-4d1e-a728-d00e746e8466 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:57.020047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:57.035684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:57.041759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75d963-71aa-4353-b6e3-bed53efdd9fb None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:57.087432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf1303-ac21-426f-b421-42e93f7aa4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:57.088209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf1303-ac21-426f-b421-42e93f7aa4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:57.088484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf1303-ac21-426f-b421-42e93f7aa4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:57.088760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf1303-ac21-426f-b421-42e93f7aa4f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:57.089442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf1303-ac21-426f-b421-42e93f7aa4f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:57.089755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf1303-ac21-426f-b421-42e93f7aa4f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:57.091530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf1303-ac21-426f-b421-42e93f7aa4f1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:57.101276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 103/187] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:57.102119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf1303-ac21-426f-b421-42e93f7aa4f1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:57.127456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b8214-dc3c-4d1e-a728-d00e746e8466 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:57.166710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d1d04-fbe7-4ba1-94e5-581a4809038a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:57.167093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d1d04-fbe7-4ba1-94e5-581a4809038a None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:57.167372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d1d04-fbe7-4ba1-94e5-581a4809038a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:57.167647 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d1d04-fbe7-4ba1-94e5-581a4809038a None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:57.168858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d1d04-fbe7-4ba1-94e5-581a4809038a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:57.170570 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d1d04-fbe7-4ba1-94e5-581a4809038a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:57.211860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf1303-ac21-426f-b421-42e93f7aa4f1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:57.245046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c1a96-06f7-4870-b368-b5e3217aa89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:57.245672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c1a96-06f7-4870-b368-b5e3217aa89a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:57.245953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c1a96-06f7-4870-b368-b5e3217aa89a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:57.246262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c1a96-06f7-4870-b368-b5e3217aa89a None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:57.247391 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c1a96-06f7-4870-b368-b5e3217aa89a None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:57.248844 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c1a96-06f7-4870-b368-b5e3217aa89a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:57.250462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 104/188] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:57 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:57.251246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c1a96-06f7-4870-b368-b5e3217aa89a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:57.280578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:57.283185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 85/189] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:45:57 2022] POST /identity/v3/users => generated 297 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:45:57.284239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d1d04-fbe7-4ba1-94e5-581a4809038a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:57.385371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:57.398594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c1a96-06f7-4870-b368-b5e3217aa89a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:57.423702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d1d04-fbe7-4ba1-94e5-581a4809038a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:57.451975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56512d65-a4c6-429c-b6f0-afd9cdf644d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:57.452989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56512d65-a4c6-429c-b6f0-afd9cdf644d2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:57.453578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56512d65-a4c6-429c-b6f0-afd9cdf644d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:57.453996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56512d65-a4c6-429c-b6f0-afd9cdf644d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:57.464591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f8968-d81e-489b-8202-a22dcba21ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:57.465630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8968-d81e-489b-8202-a22dcba21ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:57.465873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8968-d81e-489b-8202-a22dcba21ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:57.466148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8968-d81e-489b-8202-a22dcba21ff4 None admin] PATH_INFO: `/v3/users/92eb789611554f97a07d421b3eb8f039` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:57.469513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56512d65-a4c6-429c-b6f0-afd9cdf644d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-56512d65-a4c6-429c-b6f0-afd9cdf644d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:57.475227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:57.476312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:45:57.477296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:57.479433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 105/190] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:45:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:57.479953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56512d65-a4c6-429c-b6f0-afd9cdf644d2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:57.491447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f8968-d81e-489b-8202-a22dcba21ff4 None admin] RBAC: Authorizing `identity:get_user(user_id=92eb789611554f97a07d421b3eb8f039)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:57.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f8968-d81e-489b-8202-a22dcba21ff4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:57.495231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 86/191] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:57 2022] GET /identity/v3/users/92eb789611554f97a07d421b3eb8f039 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:57.495957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f8968-d81e-489b-8202-a22dcba21ff4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:57.584599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:57.598582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56512d65-a4c6-429c-b6f0-afd9cdf644d2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:57.625713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f8968-d81e-489b-8202-a22dcba21ff4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:57.643417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df28de-4031-4fd6-9f21-014e7dbcdf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:57.644152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df28de-4031-4fd6-9f21-014e7dbcdf05 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:57.644440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df28de-4031-4fd6-9f21-014e7dbcdf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:57.644765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df28de-4031-4fd6-9f21-014e7dbcdf05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:57.645511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df28de-4031-4fd6-9f21-014e7dbcdf05 None admin] RBAC: Adding query filter params (role.id=64e24908c05145b49cc926880ab769cf, scope.project.id=1231532f5f8e4acab98360e05e879050, user.id=71f957f6c8724676af13c99ddc4b62e3) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:57.645824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df28de-4031-4fd6-9f21-014e7dbcdf05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:57.648761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df28de-4031-4fd6-9f21-014e7dbcdf05 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:57.654259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 106/192] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:57 2022] GET /identity/v3/role_assignments?user.id=71f957f6c8724676af13c99ddc4b62e3&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=64e24908c05145b49cc926880ab769cf => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:57.659531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df28de-4031-4fd6-9f21-014e7dbcdf05 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:57.661818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:57.674216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918cc905-cf75-40c2-ae30-e2ec585bc0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:57.674952 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc905-cf75-40c2-ae30-e2ec585bc0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:57.675329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc905-cf75-40c2-ae30-e2ec585bc0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:57.675696 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc905-cf75-40c2-ae30-e2ec585bc0f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:57.677192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918cc905-cf75-40c2-ae30-e2ec585bc0f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:57.677606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918cc905-cf75-40c2-ae30-e2ec585bc0f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:57.679832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918cc905-cf75-40c2-ae30-e2ec585bc0f2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:57.692199 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 87/193] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:45:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:57.698580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ebdc5-79c1-414c-a703-6c7e6176e4ed None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:57.698580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ebdc5-79c1-414c-a703-6c7e6176e4ed None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:57.698580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ebdc5-79c1-414c-a703-6c7e6176e4ed None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:57.698580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 88/194] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:45:57.702085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ebdc5-79c1-414c-a703-6c7e6176e4ed None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:57.830667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df28de-4031-4fd6-9f21-014e7dbcdf05 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:57.872172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51543ec1-752b-4641-bcae-20cea5289f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:57.873384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51543ec1-752b-4641-bcae-20cea5289f73 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:57.874085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51543ec1-752b-4641-bcae-20cea5289f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:57.874512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51543ec1-752b-4641-bcae-20cea5289f73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:57.880207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ebdc5-79c1-414c-a703-6c7e6176e4ed None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:57.901323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:57.921370 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc9a8b-ecc4-4666-9ffa-0d205b629d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:57.921960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc9a8b-ecc4-4666-9ffa-0d205b629d4d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:57.922280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc9a8b-ecc4-4666-9ffa-0d205b629d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:57.922558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc9a8b-ecc4-4666-9ffa-0d205b629d4d None admin] PATH_INFO: `/v3/users/92eb789611554f97a07d421b3eb8f039` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:57.924120 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc9a8b-ecc4-4666-9ffa-0d205b629d4d None admin] RBAC: Authorizing `identity:get_user(user_id=92eb789611554f97a07d421b3eb8f039)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:57.926019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc9a8b-ecc4-4666-9ffa-0d205b629d4d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:57.928882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 89/195] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:57 2022] GET /identity/v3/users/92eb789611554f97a07d421b3eb8f039 => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:57.928882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3646630-8bc0-46ea-a03a-aa159d477723 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:57.928882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3646630-8bc0-46ea-a03a-aa159d477723 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:57.929136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3646630-8bc0-46ea-a03a-aa159d477723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:57.950015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b3646630-8bc0-46ea-a03a-aa159d477723 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:57.951265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b3646630-8bc0-46ea-a03a-aa159d477723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:45:58.011632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51543ec1-752b-4641-bcae-20cea5289f73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:58.011632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51543ec1-752b-4641-bcae-20cea5289f73 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:58.029158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:58.044344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 90/196] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:45:58.045404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3646630-8bc0-46ea-a03a-aa159d477723 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:58.134427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 107/197] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:58.141163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51543ec1-752b-4641-bcae-20cea5289f73 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:58.175032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3646630-8bc0-46ea-a03a-aa159d477723 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:58.219474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96685d-8644-4603-aee1-68930ff97d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:58.220339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96685d-8644-4603-aee1-68930ff97d69 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:58.220773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96685d-8644-4603-aee1-68930ff97d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:58.221195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96685d-8644-4603-aee1-68930ff97d69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:58.249474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:58.309857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:58.324629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96685d-8644-4603-aee1-68930ff97d69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:58.327357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96685d-8644-4603-aee1-68930ff97d69 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:58.340612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51543ec1-752b-4641-bcae-20cea5289f73 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:58.394254 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c48291-418b-433f-b7d3-5cab105ac5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:58.395031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48291-418b-433f-b7d3-5cab105ac5cc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:58.395375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48291-418b-433f-b7d3-5cab105ac5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:58.395712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48291-418b-433f-b7d3-5cab105ac5cc None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:58.397037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c48291-418b-433f-b7d3-5cab105ac5cc None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:58.398602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c48291-418b-433f-b7d3-5cab105ac5cc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:58.400848 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 108/198] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:58 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:58.401871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c48291-418b-433f-b7d3-5cab105ac5cc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:58.452636 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 91/199] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:58.453827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96685d-8644-4603-aee1-68930ff97d69 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:58.504885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c48291-418b-433f-b7d3-5cab105ac5cc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:58.524052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:58.542616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f167c-cf84-48de-83b8-e38e1f15f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:58.543508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f167c-cf84-48de-83b8-e38e1f15f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:58.543790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f167c-cf84-48de-83b8-e38e1f15f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:58.544130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f167c-cf84-48de-83b8-e38e1f15f6c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:58.557303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f167c-cf84-48de-83b8-e38e1f15f6c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-711f167c-cf84-48de-83b8-e38e1f15f6c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:45:58.559896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:45:58.560810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:45:58.561630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 109/200] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:45:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:45:58.562390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f167c-cf84-48de-83b8-e38e1f15f6c6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:58.565152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:58.582544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96685d-8644-4603-aee1-68930ff97d69 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:58.629673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36faf639-371f-40d2-9ccb-623aa25a103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:58.630641 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faf639-371f-40d2-9ccb-623aa25a103f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:58.631105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faf639-371f-40d2-9ccb-623aa25a103f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:58.631655 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faf639-371f-40d2-9ccb-623aa25a103f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:58.632974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36faf639-371f-40d2-9ccb-623aa25a103f None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d, user.id=92eb789611554f97a07d421b3eb8f039) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:58.633548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36faf639-371f-40d2-9ccb-623aa25a103f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:58.635191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36faf639-371f-40d2-9ccb-623aa25a103f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:58.643195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 92/201] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:58 2022] GET /identity/v3/role_assignments?user.id=92eb789611554f97a07d421b3eb8f039&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=62eb3e525077429aacc0ada63300802e => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:58.644261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36faf639-371f-40d2-9ccb-623aa25a103f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:58.675618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f167c-cf84-48de-83b8-e38e1f15f6c6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:58.726711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d93073-82d2-436f-97b3-a7191aa084dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:58.726711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d93073-82d2-436f-97b3-a7191aa084dc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:58.726711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d93073-82d2-436f-97b3-a7191aa084dc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:58.726711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d93073-82d2-436f-97b3-a7191aa084dc None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:58.727161 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d93073-82d2-436f-97b3-a7191aa084dc None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:58.728828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d93073-82d2-436f-97b3-a7191aa084dc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:58.731072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 110/202] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:58 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:58.737296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d93073-82d2-436f-97b3-a7191aa084dc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:58.831441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36faf639-371f-40d2-9ccb-623aa25a103f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:58.835050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:58.896345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d93073-82d2-436f-97b3-a7191aa084dc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:58.896762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b6f90-d931-4e41-b92d-27102b00173e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:58.897393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b6f90-d931-4e41-b92d-27102b00173e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:58.897678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b6f90-d931-4e41-b92d-27102b00173e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:58.897958 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b6f90-d931-4e41-b92d-27102b00173e None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:58.898635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b6f90-d931-4e41-b92d-27102b00173e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:58.898979 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b6f90-d931-4e41-b92d-27102b00173e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:58.900377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b6f90-d931-4e41-b92d-27102b00173e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:58.909164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 93/203] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:45:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:58.915186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b6f90-d931-4e41-b92d-27102b00173e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:58.940320 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:58.944254 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344e04f-a38e-45af-828c-8882ad5ef62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:58.944874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344e04f-a38e-45af-828c-8882ad5ef62b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:58.945152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344e04f-a38e-45af-828c-8882ad5ef62b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:58.945433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344e04f-a38e-45af-828c-8882ad5ef62b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:58.946167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344e04f-a38e-45af-828c-8882ad5ef62b None admin] RBAC: Adding query filter params (role.id=64e24908c05145b49cc926880ab769cf, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d, user.id=92eb789611554f97a07d421b3eb8f039) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:58.946479 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344e04f-a38e-45af-828c-8882ad5ef62b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:58.948138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344e04f-a38e-45af-828c-8882ad5ef62b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:58.965671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 111/204] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:58 2022] GET /identity/v3/role_assignments?user.id=92eb789611554f97a07d421b3eb8f039&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=64e24908c05145b49cc926880ab769cf => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:59.055823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b6f90-d931-4e41-b92d-27102b00173e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:59.094048 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509cd63-c989-48d0-9406-b31b13f52ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:59.095039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509cd63-c989-48d0-9406-b31b13f52ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:59.095520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509cd63-c989-48d0-9406-b31b13f52ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:59.095989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509cd63-c989-48d0-9406-b31b13f52ca9 None admin] PATH_INFO: `/v3/users/9eb5eb07900040518d4edbf29e170992` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:59.097501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509cd63-c989-48d0-9406-b31b13f52ca9 None admin] RBAC: Authorizing `identity:get_user(user_id=9eb5eb07900040518d4edbf29e170992)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:59.099288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509cd63-c989-48d0-9406-b31b13f52ca9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:59.101587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 94/205] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:45:58 2022] GET /identity/v3/users/9eb5eb07900040518d4edbf29e170992 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:59.107397 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344e04f-a38e-45af-828c-8882ad5ef62b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:59.255265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344e04f-a38e-45af-828c-8882ad5ef62b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:59.278290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:59.318085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d03ad-49f0-4bff-82ef-30e82088aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:59.318725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d03ad-49f0-4bff-82ef-30e82088aeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:59.319011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d03ad-49f0-4bff-82ef-30e82088aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:59.319284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d03ad-49f0-4bff-82ef-30e82088aeb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:59.404859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d03ad-49f0-4bff-82ef-30e82088aeb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:59.406257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d03ad-49f0-4bff-82ef-30e82088aeb0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:59.530871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 112/206] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:45:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:45:59.536705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509cd63-c989-48d0-9406-b31b13f52ca9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:59.658266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:59.677057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509cd63-c989-48d0-9406-b31b13f52ca9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:59.713420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9256bd3-d662-4680-8edc-fc6a172553b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:59.714076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9256bd3-d662-4680-8edc-fc6a172553b9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:59.714358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9256bd3-d662-4680-8edc-fc6a172553b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:59.714637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9256bd3-d662-4680-8edc-fc6a172553b9 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:59.716434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9256bd3-d662-4680-8edc-fc6a172553b9 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:59.717800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9256bd3-d662-4680-8edc-fc6a172553b9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:59.722332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 95/207] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:45:59 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:59.725775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d03ad-49f0-4bff-82ef-30e82088aeb0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:45:59.750674 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:45:59.873840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d03ad-49f0-4bff-82ef-30e82088aeb0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:45:59.911191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dfeb4-2139-4887-bc45-e136e6d9cee5 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:59.911502 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dfeb4-2139-4887-bc45-e136e6d9cee5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:59.911670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dfeb4-2139-4887-bc45-e136e6d9cee5 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:59.914296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 96/208] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:45:59.918415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467e7e2-b446-46c5-a7c6-7845149f9a17 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:59.923460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467e7e2-b446-46c5-a7c6-7845149f9a17 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:59.923460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467e7e2-b446-46c5-a7c6-7845149f9a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:59.923629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e36c8-c0cb-4958-8a8a-c9fcb84fc578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:45:59.923629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e36c8-c0cb-4958-8a8a-c9fcb84fc578 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:45:59.923629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e36c8-c0cb-4958-8a8a-c9fcb84fc578 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:45:59.923909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e36c8-c0cb-4958-8a8a-c9fcb84fc578 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:45:59.924639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e36c8-c0cb-4958-8a8a-c9fcb84fc578 None admin] RBAC: Adding query filter params (role.id=b6562af5bda44cc18b203749c9cc1ef3, scope.project.id=1231532f5f8e4acab98360e05e879050, user.id=9eb5eb07900040518d4edbf29e170992) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:45:59.924946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e36c8-c0cb-4958-8a8a-c9fcb84fc578 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:45:59.926566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e36c8-c0cb-4958-8a8a-c9fcb84fc578 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:45:59.935174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 113/209] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:45:59 2022] GET /identity/v3/role_assignments?user.id=9eb5eb07900040518d4edbf29e170992&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=b6562af5bda44cc18b203749c9cc1ef3 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:45:59.939309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9467e7e2-b446-46c5-a7c6-7845149f9a17 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:45:59.940784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9467e7e2-b446-46c5-a7c6-7845149f9a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:00.039440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 97/210] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:00.302247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7e8b6-2248-4f78-b70b-f5fc508d87b5 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:00.302649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7e8b6-2248-4f78-b70b-f5fc508d87b5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:00.302882 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7e8b6-2248-4f78-b70b-f5fc508d87b5 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:00.304180 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 114/211] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:00.311405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbef890-2213-40b6-a184-c2937e9ef2bb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:00.311929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbef890-2213-40b6-a184-c2937e9ef2bb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:00.312409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbef890-2213-40b6-a184-c2937e9ef2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:00.336071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9dbef890-2213-40b6-a184-c2937e9ef2bb None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:00.337558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9dbef890-2213-40b6-a184-c2937e9ef2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:00.418083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 98/212] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:00.422505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7e8b6-2248-4f78-b70b-f5fc508d87b5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:00.522145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:00.553461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7e8b6-2248-4f78-b70b-f5fc508d87b5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:00.589298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fab0fa-2d9d-4bea-94af-b6ac40aded76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:00.589979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab0fa-2d9d-4bea-94af-b6ac40aded76 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:00.590266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab0fa-2d9d-4bea-94af-b6ac40aded76 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:00.590541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab0fa-2d9d-4bea-94af-b6ac40aded76 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:00.600858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fab0fa-2d9d-4bea-94af-b6ac40aded76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-90fab0fa-2d9d-4bea-94af-b6ac40aded76 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:00.606929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:00.607827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:00.608638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 115/213] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:00.615690 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbef890-2213-40b6-a184-c2937e9ef2bb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:00.627987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:00.718648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbef890-2213-40b6-a184-c2937e9ef2bb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:00.743353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fab0fa-2d9d-4bea-94af-b6ac40aded76 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:00.761396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c1bda-f1d7-4a15-8cbd-4c55c3ad3d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:00.762217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c1bda-f1d7-4a15-8cbd-4c55c3ad3d69 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:00.762497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c1bda-f1d7-4a15-8cbd-4c55c3ad3d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:00.763107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c1bda-f1d7-4a15-8cbd-4c55c3ad3d69 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:00.763790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c1bda-f1d7-4a15-8cbd-4c55c3ad3d69 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:00.764093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c1bda-f1d7-4a15-8cbd-4c55c3ad3d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:00.765992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c1bda-f1d7-4a15-8cbd-4c55c3ad3d69 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:00.780965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 99/214] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:00.781986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c1bda-f1d7-4a15-8cbd-4c55c3ad3d69 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:00.856376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:00.872934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fab0fa-2d9d-4bea-94af-b6ac40aded76 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:00.886068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c1bda-f1d7-4a15-8cbd-4c55c3ad3d69 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:00.922957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2ac1-af3f-4831-b4c0-f660020d6746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:00.923893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2ac1-af3f-4831-b4c0-f660020d6746 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:00.924181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2ac1-af3f-4831-b4c0-f660020d6746 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:00.924460 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc2ac1-af3f-4831-b4c0-f660020d6746 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:00.925616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc2ac1-af3f-4831-b4c0-f660020d6746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:00.927226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc2ac1-af3f-4831-b4c0-f660020d6746 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:00.928078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ccee1-d7be-4440-8d0e-1453f56b2b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:00.928780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ccee1-d7be-4440-8d0e-1453f56b2b4e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:00.928899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 116/215] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:00.929062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ccee1-d7be-4440-8d0e-1453f56b2b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:00.929333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ccee1-d7be-4440-8d0e-1453f56b2b4e None admin] PATH_INFO: `/v3/users/43f74119163e4eabb8e6bde3a0ebadf2` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:00.935709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2ac1-af3f-4831-b4c0-f660020d6746 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:00.947932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ccee1-d7be-4440-8d0e-1453f56b2b4e None admin] RBAC: Authorizing `identity:get_user(user_id=43f74119163e4eabb8e6bde3a0ebadf2)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:00.949774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ccee1-d7be-4440-8d0e-1453f56b2b4e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:00.956554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 100/216] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:00 2022] GET /identity/v3/users/43f74119163e4eabb8e6bde3a0ebadf2 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:00.957644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ccee1-d7be-4440-8d0e-1453f56b2b4e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:00.991052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:01.068075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ccee1-d7be-4440-8d0e-1453f56b2b4e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:01.070663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc2ac1-af3f-4831-b4c0-f660020d6746 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:01.103168 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:01.109974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080ecad-f191-426d-8164-6f399ad7784a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:01.111017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080ecad-f191-426d-8164-6f399ad7784a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:01.118710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080ecad-f191-426d-8164-6f399ad7784a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:01.118710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080ecad-f191-426d-8164-6f399ad7784a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:01.118879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1881c-cfcf-47ac-9205-cd822be193e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:01.118879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1881c-cfcf-47ac-9205-cd822be193e3 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:01.118879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1881c-cfcf-47ac-9205-cd822be193e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:01.118879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1881c-cfcf-47ac-9205-cd822be193e3 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:01.119387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1881c-cfcf-47ac-9205-cd822be193e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:01.121138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1881c-cfcf-47ac-9205-cd822be193e3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:01.184006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080ecad-f191-426d-8164-6f399ad7784a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:01.185474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080ecad-f191-426d-8164-6f399ad7784a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:01.246294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 117/217] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:46:00 2022] POST /identity/v3/users => generated 297 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:46:01.246294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1d26-b761-4389-a7d6-6145eb047663 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:01.246294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1d26-b761-4389-a7d6-6145eb047663 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:01.246294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1d26-b761-4389-a7d6-6145eb047663 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:01.246294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 118/218] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:01.251346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba72075-92c8-47d9-b77c-11a2ad503181 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:01.251552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba72075-92c8-47d9-b77c-11a2ad503181 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:01.251813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba72075-92c8-47d9-b77c-11a2ad503181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:01.271425 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6ba72075-92c8-47d9-b77c-11a2ad503181 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:01.272667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6ba72075-92c8-47d9-b77c-11a2ad503181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:01.312992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 101/219] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:01.314315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080ecad-f191-426d-8164-6f399ad7784a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:01.337362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:01.352957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 119/220] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:01.355849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:01.439819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080ecad-f191-426d-8164-6f399ad7784a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:01.482059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67493c6-134c-4b5f-8c60-dc36390f3a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:01.482717 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67493c6-134c-4b5f-8c60-dc36390f3a16 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:01.482999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67493c6-134c-4b5f-8c60-dc36390f3a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:01.483274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67493c6-134c-4b5f-8c60-dc36390f3a16 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:01.484416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67493c6-134c-4b5f-8c60-dc36390f3a16 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:01.485876 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67493c6-134c-4b5f-8c60-dc36390f3a16 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:01.487555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 102/221] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:01 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:01.493689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba72075-92c8-47d9-b77c-11a2ad503181 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:01.555714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d96bd-463d-4cce-acbb-a722e1f01718 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:01.556057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d96bd-463d-4cce-acbb-a722e1f01718 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:01.556276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d96bd-463d-4cce-acbb-a722e1f01718 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:01.557623 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 103/222] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:01.563071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5178a-c009-4a30-bdeb-514ecbfb7dec None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:01.563351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5178a-c009-4a30-bdeb-514ecbfb7dec None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:01.563630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5178a-c009-4a30-bdeb-514ecbfb7dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:01.590974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1cb5178a-c009-4a30-bdeb-514ecbfb7dec None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:01.593224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1cb5178a-c009-4a30-bdeb-514ecbfb7dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:01.614614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba72075-92c8-47d9-b77c-11a2ad503181 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:01.656807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce864962-4f80-4cb2-8c29-bfd64e98475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:01.657439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce864962-4f80-4cb2-8c29-bfd64e98475e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:01.657724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce864962-4f80-4cb2-8c29-bfd64e98475e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:01.658001 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce864962-4f80-4cb2-8c29-bfd64e98475e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:01.658793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce864962-4f80-4cb2-8c29-bfd64e98475e None admin] RBAC: Adding query filter params (role.id=991609c9013b45b3907788ef9b42d051, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d, user.id=43f74119163e4eabb8e6bde3a0ebadf2) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:01.659789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce864962-4f80-4cb2-8c29-bfd64e98475e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:01.660949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce864962-4f80-4cb2-8c29-bfd64e98475e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:01.669961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 120/223] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:01 2022] GET /identity/v3/role_assignments?user.id=43f74119163e4eabb8e6bde3a0ebadf2&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=991609c9013b45b3907788ef9b42d051 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:01.670624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce864962-4f80-4cb2-8c29-bfd64e98475e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:01.687618 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 104/224] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:46:01.734655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:01.796355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce864962-4f80-4cb2-8c29-bfd64e98475e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:01.833181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddf460-7aee-4a34-9fcb-c176107e3e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:01.833819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddf460-7aee-4a34-9fcb-c176107e3e97 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:01.834095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddf460-7aee-4a34-9fcb-c176107e3e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:01.834362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddf460-7aee-4a34-9fcb-c176107e3e97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:01.845433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddf460-7aee-4a34-9fcb-c176107e3e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-8bddf460-7aee-4a34-9fcb-c176107e3e97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:01.850937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:01.851824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:01.853385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 121/225] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:46:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:01.858050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5178a-c009-4a30-bdeb-514ecbfb7dec None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:01.889853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:01.973251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5178a-c009-4a30-bdeb-514ecbfb7dec None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:02.013030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e40c3e-e20d-425c-bd23-29120f75eedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:02.013568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e40c3e-e20d-425c-bd23-29120f75eedc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:02.013847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e40c3e-e20d-425c-bd23-29120f75eedc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:02.014114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e40c3e-e20d-425c-bd23-29120f75eedc None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:02.014813 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e40c3e-e20d-425c-bd23-29120f75eedc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:02.015113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e40c3e-e20d-425c-bd23-29120f75eedc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:02.016562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e40c3e-e20d-425c-bd23-29120f75eedc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:02.025595 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 105/226] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:02.031138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddf460-7aee-4a34-9fcb-c176107e3e97 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:02.039610 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e40c3e-e20d-425c-bd23-29120f75eedc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:02.150189 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddf460-7aee-4a34-9fcb-c176107e3e97 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:02.161449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:02.167270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:02.181587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e40c3e-e20d-425c-bd23-29120f75eedc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:02.195980 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e5fa3-9fbd-4055-a473-9a4f9360b9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:02.196958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5fa3-9fbd-4055-a473-9a4f9360b9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:02.197433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5fa3-9fbd-4055-a473-9a4f9360b9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:02.197948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5fa3-9fbd-4055-a473-9a4f9360b9c8 None admin] PATH_INFO: `/v3/users/92eb789611554f97a07d421b3eb8f039` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:02.199565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e5fa3-9fbd-4055-a473-9a4f9360b9c8 None admin] RBAC: Authorizing `identity:get_user(user_id=92eb789611554f97a07d421b3eb8f039)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:02.201719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e5fa3-9fbd-4055-a473-9a4f9360b9c8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:02.204295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 122/227] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:02 2022] GET /identity/v3/users/92eb789611554f97a07d421b3eb8f039 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:02.209914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e5fa3-9fbd-4055-a473-9a4f9360b9c8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:02.253164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e49047-0258-43f4-97c2-ef707f988bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:02.254208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e49047-0258-43f4-97c2-ef707f988bea None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:02.255298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e49047-0258-43f4-97c2-ef707f988bea None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:02.255785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e49047-0258-43f4-97c2-ef707f988bea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:02.268353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e49047-0258-43f4-97c2-ef707f988bea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-88e49047-0258-43f4-97c2-ef707f988bea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:02.274342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:02.275323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:02.276119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:02.279020 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 106/228] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:46:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:02.284877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e49047-0258-43f4-97c2-ef707f988bea None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:02.399862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e49047-0258-43f4-97c2-ef707f988bea None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:02.457481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e5fa3-9fbd-4055-a473-9a4f9360b9c8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:02.492302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5c905-ed77-41d1-9067-22e579ee137b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:02.492934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5c905-ed77-41d1-9067-22e579ee137b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:02.493215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5c905-ed77-41d1-9067-22e579ee137b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:02.493490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5c905-ed77-41d1-9067-22e579ee137b None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:02.494170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5c905-ed77-41d1-9067-22e579ee137b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:02.494512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5c905-ed77-41d1-9067-22e579ee137b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:02.495895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0d4d3-11df-4b15-8750-17d4f18b8aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:02.496551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5c905-ed77-41d1-9067-22e579ee137b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:02.497400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0d4d3-11df-4b15-8750-17d4f18b8aef None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:02.498006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0d4d3-11df-4b15-8750-17d4f18b8aef None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:02.498482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0d4d3-11df-4b15-8750-17d4f18b8aef None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:02.499986 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0d4d3-11df-4b15-8750-17d4f18b8aef None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:02.501643 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0d4d3-11df-4b15-8750-17d4f18b8aef None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:02.503858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 123/229] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:02 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:02.510879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 107/230] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:02.512677 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0d4d3-11df-4b15-8750-17d4f18b8aef None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:02.529973 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5c905-ed77-41d1-9067-22e579ee137b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:02.571078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:02.619119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:02.627642 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0d4d3-11df-4b15-8750-17d4f18b8aef None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:02.677633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8da47-aa56-4129-94a6-2185378fc356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:02.678704 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8da47-aa56-4129-94a6-2185378fc356 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:02.679203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8da47-aa56-4129-94a6-2185378fc356 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:02.679744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8da47-aa56-4129-94a6-2185378fc356 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/users/92eb789611554f97a07d421b3eb8f039/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:02.688484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8da47-aa56-4129-94a6-2185378fc356 None admin] RBAC: Authorizing `identity:create_grant(project_id=d64c95c677b44381b0ee5c5ab3c1d15d, user_id=92eb789611554f97a07d421b3eb8f039, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:02.690524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8da47-aa56-4129-94a6-2185378fc356 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:02.712379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5c905-ed77-41d1-9067-22e579ee137b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:02.747501 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 124/231] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:46:02 2022] PUT /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/users/92eb789611554f97a07d421b3eb8f039/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:02.769848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138614c-d7fa-4472-a065-01d134dd333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:02.770519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138614c-d7fa-4472-a065-01d134dd333b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:02.770990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138614c-d7fa-4472-a065-01d134dd333b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:02.771261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138614c-d7fa-4472-a065-01d134dd333b None admin] PATH_INFO: `/v3/users/92eb789611554f97a07d421b3eb8f039` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:02.772605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138614c-d7fa-4472-a065-01d134dd333b None admin] RBAC: Authorizing `identity:get_user(user_id=92eb789611554f97a07d421b3eb8f039)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:02.774209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138614c-d7fa-4472-a065-01d134dd333b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:02.776362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 108/232] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:02 2022] GET /identity/v3/users/92eb789611554f97a07d421b3eb8f039 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:02.782186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8da47-aa56-4129-94a6-2185378fc356 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:02.911591 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8da47-aa56-4129-94a6-2185378fc356 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:02.931092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:02.963394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddbeac-7aa3-4fb4-93e4-d7aa90e75201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:02.964051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddbeac-7aa3-4fb4-93e4-d7aa90e75201 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:02.964333 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddbeac-7aa3-4fb4-93e4-d7aa90e75201 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:02.964699 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddbeac-7aa3-4fb4-93e4-d7aa90e75201 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:02.966900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddbeac-7aa3-4fb4-93e4-d7aa90e75201 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:02.969053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddbeac-7aa3-4fb4-93e4-d7aa90e75201 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:02.970932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 125/233] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:02 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:02.977639 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138614c-d7fa-4472-a065-01d134dd333b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:03.067354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:03.086729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138614c-d7fa-4472-a065-01d134dd333b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:03.134143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16158f3-c18e-467d-9a44-c865e734c54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:03.134844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16158f3-c18e-467d-9a44-c865e734c54f None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:03.135125 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16158f3-c18e-467d-9a44-c865e734c54f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:03.135396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16158f3-c18e-467d-9a44-c865e734c54f None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/users/92eb789611554f97a07d421b3eb8f039/roles/64e24908c05145b49cc926880ab769cf` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:03.137663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16158f3-c18e-467d-9a44-c865e734c54f None admin] RBAC: Authorizing `identity:create_grant(project_id=d64c95c677b44381b0ee5c5ab3c1d15d, user_id=92eb789611554f97a07d421b3eb8f039, role_id=64e24908c05145b49cc926880ab769cf)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:03.139163 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16158f3-c18e-467d-9a44-c865e734c54f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:03.236372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 109/234] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:46:02 2022] PUT /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/users/92eb789611554f97a07d421b3eb8f039/roles/64e24908c05145b49cc926880ab769cf => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:03.952606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e32d7c-316b-4510-940b-5fc96b1c5cbf None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:03.953122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e32d7c-316b-4510-940b-5fc96b1c5cbf None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:03.953122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e32d7c-316b-4510-940b-5fc96b1c5cbf None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:03.954802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 126/235] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:03.960972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab0c09-1b4a-488e-be1b-3f9aa6062657 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:03.961541 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab0c09-1b4a-488e-be1b-3f9aa6062657 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:03.962024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab0c09-1b4a-488e-be1b-3f9aa6062657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:03.989959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-eeab0c09-1b4a-488e-be1b-3f9aa6062657 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:03.991129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eeab0c09-1b4a-488e-be1b-3f9aa6062657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:04.085283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 110/236] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:46:04.354235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738476e-eb02-4698-93c1-cfe6348b9ead None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:04.354940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738476e-eb02-4698-93c1-cfe6348b9ead None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:04.355964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738476e-eb02-4698-93c1-cfe6348b9ead None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:04.357960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 127/237] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:04.362907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c7ed-6b50-4fb6-a645-ea7bc89c2201 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:04.363189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c7ed-6b50-4fb6-a645-ea7bc89c2201 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:04.363461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c7ed-6b50-4fb6-a645-ea7bc89c2201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:04.384890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d9a8c7ed-6b50-4fb6-a645-ea7bc89c2201 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:04.386069 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d9a8c7ed-6b50-4fb6-a645-ea7bc89c2201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:04.435132 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d043bf2-8dca-4831-bea3-d463589827d1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:04.435461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d043bf2-8dca-4831-bea3-d463589827d1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:04.435694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d043bf2-8dca-4831-bea3-d463589827d1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:04.437285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 128/238] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:04.442762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7824d2f-405b-4938-97e9-82a55ffaac6d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:04.443054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7824d2f-405b-4938-97e9-82a55ffaac6d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:04.443337 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7824d2f-405b-4938-97e9-82a55ffaac6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:04.474653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 111/239] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:04.475298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:04.479787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f7824d2f-405b-4938-97e9-82a55ffaac6d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:04.481532 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f7824d2f-405b-4938-97e9-82a55ffaac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:04.547411 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8c7ed-6b50-4fb6-a645-ea7bc89c2201 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:04.592023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 129/240] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:46:04.691040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8c7ed-6b50-4fb6-a645-ea7bc89c2201 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:04.734360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2baf82-c477-4bb6-b1b0-5f9c6992988f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:04.735035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2baf82-c477-4bb6-b1b0-5f9c6992988f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:04.737744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2baf82-c477-4bb6-b1b0-5f9c6992988f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:04.737744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2baf82-c477-4bb6-b1b0-5f9c6992988f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:04.748616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2baf82-c477-4bb6-b1b0-5f9c6992988f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5e2baf82-c477-4bb6-b1b0-5f9c6992988f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:04.753463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:04.754338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:04.755158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 112/241] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:04.760497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7824d2f-405b-4938-97e9-82a55ffaac6d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:04.779614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2baf82-c477-4bb6-b1b0-5f9c6992988f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:04.878481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:04.925585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7824d2f-405b-4938-97e9-82a55ffaac6d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:04.932967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2baf82-c477-4bb6-b1b0-5f9c6992988f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:04.968638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f5f5d-d3c0-4ec0-963f-5ff6dc0489c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:04.969203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f5f5d-d3c0-4ec0-963f-5ff6dc0489c5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:04.969486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f5f5d-d3c0-4ec0-963f-5ff6dc0489c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:04.969763 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f5f5d-d3c0-4ec0-963f-5ff6dc0489c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:04.970932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f5f5d-d3c0-4ec0-963f-5ff6dc0489c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:04.972408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f5f5d-d3c0-4ec0-963f-5ff6dc0489c5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:04.974142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 113/242] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:04.978789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f5f5d-d3c0-4ec0-963f-5ff6dc0489c5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:04.999622 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:05.031431 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fa08f-850e-48bf-b4b3-f6c407d61add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:05.032436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fa08f-850e-48bf-b4b3-f6c407d61add None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:05.032924 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fa08f-850e-48bf-b4b3-f6c407d61add None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:05.033410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fa08f-850e-48bf-b4b3-f6c407d61add None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:05.035272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fa08f-850e-48bf-b4b3-f6c407d61add None admin] RBAC: Adding query filter params (name=reader) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:05.035804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fa08f-850e-48bf-b4b3-f6c407d61add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:05.037405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fa08f-850e-48bf-b4b3-f6c407d61add None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:05.047611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 130/243] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:05.049026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fa08f-850e-48bf-b4b3-f6c407d61add None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:05.110234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f5f5d-d3c0-4ec0-963f-5ff6dc0489c5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:05.146508 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:05.159517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7a4b0-c8ea-4131-95ca-81764e19c810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:05.161100 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7a4b0-c8ea-4131-95ca-81764e19c810 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:05.161585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7a4b0-c8ea-4131-95ca-81764e19c810 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:05.162059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7a4b0-c8ea-4131-95ca-81764e19c810 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:05.162794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fa08f-850e-48bf-b4b3-f6c407d61add None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:05.175819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7a4b0-c8ea-4131-95ca-81764e19c810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3db7a4b0-c8ea-4131-95ca-81764e19c810 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:05.183462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:05.184334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:05.185135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:05.188223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 114/244] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:05.189431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7a4b0-c8ea-4131-95ca-81764e19c810 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:05.204540 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d61a97-199d-40cf-b008-e10adb16b54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:05.205604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61a97-199d-40cf-b008-e10adb16b54e None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:05.206075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61a97-199d-40cf-b008-e10adb16b54e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:05.206542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d61a97-199d-40cf-b008-e10adb16b54e None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:05.207694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d61a97-199d-40cf-b008-e10adb16b54e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:05.209449 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d61a97-199d-40cf-b008-e10adb16b54e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:05.286426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 131/245] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:46:05 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:46:05.295856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d61a97-199d-40cf-b008-e10adb16b54e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:05.320699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7a4b0-c8ea-4131-95ca-81764e19c810 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:05.338388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:05.376374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54fe81-c64a-4927-af88-a531b2b95aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:05.377391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54fe81-c64a-4927-af88-a531b2b95aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:05.377977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54fe81-c64a-4927-af88-a531b2b95aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:05.378401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54fe81-c64a-4927-af88-a531b2b95aa9 None admin] PATH_INFO: `/v3/users/fdac03db82fc4fbca4211d421e0c5be9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:05.398833 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54fe81-c64a-4927-af88-a531b2b95aa9 None admin] RBAC: Authorizing `identity:get_user(user_id=fdac03db82fc4fbca4211d421e0c5be9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:05.400631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54fe81-c64a-4927-af88-a531b2b95aa9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:05.404634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 115/246] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:05 2022] GET /identity/v3/users/fdac03db82fc4fbca4211d421e0c5be9 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:05.406534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3f6ed-9600-4149-b08d-94239ea19faf None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:05.408798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3f6ed-9600-4149-b08d-94239ea19faf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:05.408798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3f6ed-9600-4149-b08d-94239ea19faf None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:05.409217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 116/247] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:05.409950 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3f6ed-9600-4149-b08d-94239ea19faf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:05.430791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d61a97-199d-40cf-b008-e10adb16b54e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:05.470939 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfea3e-679a-450c-bb4c-20bdbfa7f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:05.471956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfea3e-679a-450c-bb4c-20bdbfa7f54d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:05.472444 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfea3e-679a-450c-bb4c-20bdbfa7f54d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:05.472959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfea3e-679a-450c-bb4c-20bdbfa7f54d None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:05.473892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cfea3e-679a-450c-bb4c-20bdbfa7f54d None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:05.474405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cfea3e-679a-450c-bb4c-20bdbfa7f54d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:05.476164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cfea3e-679a-450c-bb4c-20bdbfa7f54d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:05.477228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:05.487226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 132/248] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:05.488968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7da35-7281-4ea9-87e3-2148ae7b3790 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:05.489457 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7da35-7281-4ea9-87e3-2148ae7b3790 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:05.489929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7da35-7281-4ea9-87e3-2148ae7b3790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:05.510903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-80d7da35-7281-4ea9-87e3-2148ae7b3790 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:05.512464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-80d7da35-7281-4ea9-87e3-2148ae7b3790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:05.550511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3f6ed-9600-4149-b08d-94239ea19faf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:05.600780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ac5d6-0aa4-46f8-8e75-b175ca266fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:05.601607 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 133/249] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:05.602063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ac5d6-0aa4-46f8-8e75-b175ca266fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:05.602571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ac5d6-0aa4-46f8-8e75-b175ca266fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:05.603030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ac5d6-0aa4-46f8-8e75-b175ca266fd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:05.603624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7da35-7281-4ea9-87e3-2148ae7b3790 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:05.635122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:05.691615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ac5d6-0aa4-46f8-8e75-b175ca266fd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:05.693276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ac5d6-0aa4-46f8-8e75-b175ca266fd9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:05.745310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7da35-7281-4ea9-87e3-2148ae7b3790 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:05.752893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:05.795415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7566b0c-bdad-4d54-ab05-b1e476323020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:05.796311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7566b0c-bdad-4d54-ab05-b1e476323020 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:05.797073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7566b0c-bdad-4d54-ab05-b1e476323020 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:05.797982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7566b0c-bdad-4d54-ab05-b1e476323020 None admin] PATH_INFO: `/v3/users/43f74119163e4eabb8e6bde3a0ebadf2` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:05.799569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7566b0c-bdad-4d54-ab05-b1e476323020 None admin] RBAC: Authorizing `identity:get_user(user_id=43f74119163e4eabb8e6bde3a0ebadf2)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:05.801225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7566b0c-bdad-4d54-ab05-b1e476323020 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:05.804015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 134/250] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:05 2022] GET /identity/v3/users/43f74119163e4eabb8e6bde3a0ebadf2 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:05.810829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7566b0c-bdad-4d54-ab05-b1e476323020 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:05.824739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 117/251] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:05.831144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ac5d6-0aa4-46f8-8e75-b175ca266fd9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:05.930510 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:05.950283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7566b0c-bdad-4d54-ab05-b1e476323020 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:05.967842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ac5d6-0aa4-46f8-8e75-b175ca266fd9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:05.991183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d7270-7a1a-4c6e-9baf-197d9652416d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:05.991829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d7270-7a1a-4c6e-9baf-197d9652416d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:05.992110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d7270-7a1a-4c6e-9baf-197d9652416d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:05.992385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d7270-7a1a-4c6e-9baf-197d9652416d None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:05.993551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d7270-7a1a-4c6e-9baf-197d9652416d None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:05.995033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d7270-7a1a-4c6e-9baf-197d9652416d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:05.997082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 135/252] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:05 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:05.997916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d7270-7a1a-4c6e-9baf-197d9652416d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:06.041137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d9be4-2595-4e0c-bc37-5ca96a497867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:06.042212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d9be4-2595-4e0c-bc37-5ca96a497867 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:06.042881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d9be4-2595-4e0c-bc37-5ca96a497867 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:06.043362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d9be4-2595-4e0c-bc37-5ca96a497867 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:06.047511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d9be4-2595-4e0c-bc37-5ca96a497867 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:06.049346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d9be4-2595-4e0c-bc37-5ca96a497867 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:06.051765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 118/253] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:05 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:06.052881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d9be4-2595-4e0c-bc37-5ca96a497867 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:06.084000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:06.110988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d7270-7a1a-4c6e-9baf-197d9652416d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:06.150311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fc6d6-afdb-4d33-b85b-2f2bae5ab338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:06.151253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fc6d6-afdb-4d33-b85b-2f2bae5ab338 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:06.151534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fc6d6-afdb-4d33-b85b-2f2bae5ab338 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:06.151812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fc6d6-afdb-4d33-b85b-2f2bae5ab338 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:06.168753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fc6d6-afdb-4d33-b85b-2f2bae5ab338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-703fc6d6-afdb-4d33-b85b-2f2bae5ab338 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:06.175559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:06.176464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:06.177258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 136/254] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:46:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:06.177916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fc6d6-afdb-4d33-b85b-2f2bae5ab338 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:06.197107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d9be4-2595-4e0c-bc37-5ca96a497867 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:06.199610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:06.231622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de643f-21ec-4ed8-8920-5cf7359da92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:06.232298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de643f-21ec-4ed8-8920-5cf7359da92b None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:06.232585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de643f-21ec-4ed8-8920-5cf7359da92b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:06.232869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de643f-21ec-4ed8-8920-5cf7359da92b None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/users/43f74119163e4eabb8e6bde3a0ebadf2/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:06.234966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de643f-21ec-4ed8-8920-5cf7359da92b None admin] RBAC: Authorizing `identity:create_grant(project_id=d64c95c677b44381b0ee5c5ab3c1d15d, user_id=43f74119163e4eabb8e6bde3a0ebadf2, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:06.236515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de643f-21ec-4ed8-8920-5cf7359da92b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:06.303936 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fc6d6-afdb-4d33-b85b-2f2bae5ab338 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:06.307401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 119/255] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:46:06 2022] PUT /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/users/43f74119163e4eabb8e6bde3a0ebadf2/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:06.309341 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad07219-f75d-4902-8c5b-9b156eef8792 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:06.309622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad07219-f75d-4902-8c5b-9b156eef8792 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:06.309893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad07219-f75d-4902-8c5b-9b156eef8792 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:06.311466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 120/256] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:06.312218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad07219-f75d-4902-8c5b-9b156eef8792 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:06.373441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96849aa9-df6d-4c38-80aa-bec2a021f418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:06.374210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96849aa9-df6d-4c38-80aa-bec2a021f418 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:06.374495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96849aa9-df6d-4c38-80aa-bec2a021f418 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:06.374823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96849aa9-df6d-4c38-80aa-bec2a021f418 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:06.375737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96849aa9-df6d-4c38-80aa-bec2a021f418 None admin] RBAC: Adding query filter params (role.id=b6562af5bda44cc18b203749c9cc1ef3, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d, user.id=fdac03db82fc4fbca4211d421e0c5be9) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:06.376108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96849aa9-df6d-4c38-80aa-bec2a021f418 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:06.377720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96849aa9-df6d-4c38-80aa-bec2a021f418 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:06.391943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 137/257] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:06 2022] GET /identity/v3/role_assignments?user.id=fdac03db82fc4fbca4211d421e0c5be9&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=b6562af5bda44cc18b203749c9cc1ef3 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:06.393460 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606d9b2-718f-4186-a765-5fe4c0e80c8b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:06.393751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606d9b2-718f-4186-a765-5fe4c0e80c8b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:06.394029 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606d9b2-718f-4186-a765-5fe4c0e80c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:06.407134 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:06.430127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1606d9b2-718f-4186-a765-5fe4c0e80c8b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:06.431394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1606d9b2-718f-4186-a765-5fe4c0e80c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:06.438533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad07219-f75d-4902-8c5b-9b156eef8792 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:06.484319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabec91-78d1-47ff-8e40-31ac67978df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:06.484963 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabec91-78d1-47ff-8e40-31ac67978df5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:06.485340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabec91-78d1-47ff-8e40-31ac67978df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:06.485524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabec91-78d1-47ff-8e40-31ac67978df5 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:06.486942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabec91-78d1-47ff-8e40-31ac67978df5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:06.487544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabec91-78d1-47ff-8e40-31ac67978df5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:06.489335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabec91-78d1-47ff-8e40-31ac67978df5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:06.501499 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 121/258] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:06.507463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabec91-78d1-47ff-8e40-31ac67978df5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:06.587700 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 138/259] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:06.631616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabec91-78d1-47ff-8e40-31ac67978df5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:06.668987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6766be9-9c49-406b-a712-4e6e5137aae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:06.669652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6766be9-9c49-406b-a712-4e6e5137aae1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:06.669929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6766be9-9c49-406b-a712-4e6e5137aae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:06.670200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6766be9-9c49-406b-a712-4e6e5137aae1 None admin] PATH_INFO: `/v3/users/92eb789611554f97a07d421b3eb8f039` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:06.671428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6766be9-9c49-406b-a712-4e6e5137aae1 None admin] RBAC: Authorizing `identity:get_user(user_id=92eb789611554f97a07d421b3eb8f039)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:06.672915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6766be9-9c49-406b-a712-4e6e5137aae1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:06.674912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 122/260] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:06 2022] GET /identity/v3/users/92eb789611554f97a07d421b3eb8f039 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:06.682105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606d9b2-718f-4186-a765-5fe4c0e80c8b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:06.699570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:06.819947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606d9b2-718f-4186-a765-5fe4c0e80c8b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:06.870224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130547c-bb8a-4eb1-acc9-f5f70b425aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:06.871416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130547c-bb8a-4eb1-acc9-f5f70b425aed None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:06.871707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130547c-bb8a-4eb1-acc9-f5f70b425aed None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:06.871983 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130547c-bb8a-4eb1-acc9-f5f70b425aed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:06.948256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6766be9-9c49-406b-a712-4e6e5137aae1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:06.958046 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:06.970181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130547c-bb8a-4eb1-acc9-f5f70b425aed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:06.971684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130547c-bb8a-4eb1-acc9-f5f70b425aed None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:06.985645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:07.068217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6766be9-9c49-406b-a712-4e6e5137aae1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:07.134333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4e628-8715-4482-ad8a-34f02712aed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:07.134911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4e628-8715-4482-ad8a-34f02712aed1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:07.135188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4e628-8715-4482-ad8a-34f02712aed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:07.135455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4e628-8715-4482-ad8a-34f02712aed1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:07.157274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4e628-8715-4482-ad8a-34f02712aed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-86a4e628-8715-4482-ad8a-34f02712aed1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:07.164060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:07.164937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:07.165729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 123/261] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:46:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:07.174023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 139/262] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:07.177180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4e628-8715-4482-ad8a-34f02712aed1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:07.179466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130547c-bb8a-4eb1-acc9-f5f70b425aed None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:07.288616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130547c-bb8a-4eb1-acc9-f5f70b425aed None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:07.295687 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:07.297371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:07.327689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8b5b4-a687-405c-b251-3bb7717cb417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:07.328337 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8b5b4-a687-405c-b251-3bb7717cb417 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:07.328615 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8b5b4-a687-405c-b251-3bb7717cb417 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:07.328889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8b5b4-a687-405c-b251-3bb7717cb417 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:07.330022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8b5b4-a687-405c-b251-3bb7717cb417 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:07.331565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8b5b4-a687-405c-b251-3bb7717cb417 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:07.333560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 140/263] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:07 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:07.347833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8b5b4-a687-405c-b251-3bb7717cb417 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:07.374106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4e628-8715-4482-ad8a-34f02712aed1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:07.471377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd4b5-0a1c-4bcf-8df0-28cd8681e630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:07.472398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd4b5-0a1c-4bcf-8df0-28cd8681e630 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:07.472894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd4b5-0a1c-4bcf-8df0-28cd8681e630 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:07.473383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd4b5-0a1c-4bcf-8df0-28cd8681e630 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:07.474267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bd4b5-0a1c-4bcf-8df0-28cd8681e630 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:07.474805 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bd4b5-0a1c-4bcf-8df0-28cd8681e630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:07.476456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bd4b5-0a1c-4bcf-8df0-28cd8681e630 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:07.477716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8b5b4-a687-405c-b251-3bb7717cb417 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:07.494047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 124/264] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:07.499413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd4b5-0a1c-4bcf-8df0-28cd8681e630 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:07.518706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fc5e1-d981-48f3-a68e-ab02f0b2ae54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:07.518706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fc5e1-d981-48f3-a68e-ab02f0b2ae54 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:07.518706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fc5e1-d981-48f3-a68e-ab02f0b2ae54 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:07.518706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fc5e1-d981-48f3-a68e-ab02f0b2ae54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:07.519095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fc5e1-d981-48f3-a68e-ab02f0b2ae54 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d, user.id=92eb789611554f97a07d421b3eb8f039) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:07.519095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fc5e1-d981-48f3-a68e-ab02f0b2ae54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:07.519095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fc5e1-d981-48f3-a68e-ab02f0b2ae54 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:07.523963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 141/265] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:07 2022] GET /identity/v3/role_assignments?user.id=92eb789611554f97a07d421b3eb8f039&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=62eb3e525077429aacc0ada63300802e => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:07.668261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd4b5-0a1c-4bcf-8df0-28cd8681e630 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:07.700998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:07.711362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b472d-e5dd-41da-8aad-0b0f65bbea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:07.712009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b472d-e5dd-41da-8aad-0b0f65bbea1a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:07.712291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b472d-e5dd-41da-8aad-0b0f65bbea1a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:07.712566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b472d-e5dd-41da-8aad-0b0f65bbea1a None admin] PATH_INFO: `/v3/users/92eb789611554f97a07d421b3eb8f039` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:07.713711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b472d-e5dd-41da-8aad-0b0f65bbea1a None admin] RBAC: Authorizing `identity:get_user(user_id=92eb789611554f97a07d421b3eb8f039)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:07.715249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b472d-e5dd-41da-8aad-0b0f65bbea1a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:07.720837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 125/266] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:07 2022] GET /identity/v3/users/92eb789611554f97a07d421b3eb8f039 => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:07.724148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fc5e1-d981-48f3-a68e-ab02f0b2ae54 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:07.769830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:07.833348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fc5e1-d981-48f3-a68e-ab02f0b2ae54 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:07.869136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16943f9-95f8-430b-8018-8238f96c4c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:07.869759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16943f9-95f8-430b-8018-8238f96c4c84 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:07.870042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16943f9-95f8-430b-8018-8238f96c4c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:07.870320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16943f9-95f8-430b-8018-8238f96c4c84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:07.942287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16943f9-95f8-430b-8018-8238f96c4c84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:07.943788 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16943f9-95f8-430b-8018-8238f96c4c84 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:07.986298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:08.050407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 142/267] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 07:46:08.059995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b472d-e5dd-41da-8aad-0b0f65bbea1a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:08.195927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b472d-e5dd-41da-8aad-0b0f65bbea1a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:08.239658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbeaa1-ee7f-4870-8891-db89f20d2087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:08.240216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbeaa1-ee7f-4870-8891-db89f20d2087 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:08.240495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbeaa1-ee7f-4870-8891-db89f20d2087 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:08.240767 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbeaa1-ee7f-4870-8891-db89f20d2087 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:08.247428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbeaa1-ee7f-4870-8891-db89f20d2087 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:08.248913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbeaa1-ee7f-4870-8891-db89f20d2087 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:08.250813 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 126/268] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:08 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:08.256885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16943f9-95f8-430b-8018-8238f96c4c84 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:08.368766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16943f9-95f8-430b-8018-8238f96c4c84 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:08.419619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e8a91-a7b7-4971-b877-450bce709176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:08.420254 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e8a91-a7b7-4971-b877-450bce709176 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:08.420527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e8a91-a7b7-4971-b877-450bce709176 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:08.420805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e8a91-a7b7-4971-b877-450bce709176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:08.421555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e8a91-a7b7-4971-b877-450bce709176 None admin] RBAC: Adding query filter params (role.id=64e24908c05145b49cc926880ab769cf, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d, user.id=92eb789611554f97a07d421b3eb8f039) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:08.421866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e8a91-a7b7-4971-b877-450bce709176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:08.424061 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e8a91-a7b7-4971-b877-450bce709176 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:08.437409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 143/269] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:08 2022] GET /identity/v3/role_assignments?user.id=92eb789611554f97a07d421b3eb8f039&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=64e24908c05145b49cc926880ab769cf => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:08.607865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d195b7e-dc25-48b3-bd61-46957689bb07 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:08.608852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d195b7e-dc25-48b3-bd61-46957689bb07 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:08.609349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d195b7e-dc25-48b3-bd61-46957689bb07 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:08.611299 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 127/270] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:08.617550 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8687e95-f2d3-43ea-a0b0-0ef52595f8f1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:08.617840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8687e95-f2d3-43ea-a0b0-0ef52595f8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:08.618120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8687e95-f2d3-43ea-a0b0-0ef52595f8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:08.625849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:08.643475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e8687e95-f2d3-43ea-a0b0-0ef52595f8f1 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:08.644735 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e8687e95-f2d3-43ea-a0b0-0ef52595f8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:08.729968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 144/271] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:08.786382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fa507-013f-4637-b607-2f989a348351 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:08.787007 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fa507-013f-4637-b607-2f989a348351 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:08.787478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fa507-013f-4637-b607-2f989a348351 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:08.789319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 128/272] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:08.795534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f53e8b-d1ca-430a-8cdf-36d04d7dbbfc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:08.795825 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f53e8b-d1ca-430a-8cdf-36d04d7dbbfc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:08.796102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f53e8b-d1ca-430a-8cdf-36d04d7dbbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:08.816024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-17f53e8b-d1ca-430a-8cdf-36d04d7dbbfc None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:08.817220 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-17f53e8b-d1ca-430a-8cdf-36d04d7dbbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:08.895652 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 145/273] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:09.043946 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fa507-013f-4637-b607-2f989a348351 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:09.117538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:09.164096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fa507-013f-4637-b607-2f989a348351 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:09.203382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df2d23-2943-41b7-98aa-10ca6eaec9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:09.204392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df2d23-2943-41b7-98aa-10ca6eaec9fa None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:09.204865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df2d23-2943-41b7-98aa-10ca6eaec9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:09.205332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df2d23-2943-41b7-98aa-10ca6eaec9fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:09.207980 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f53e8b-d1ca-430a-8cdf-36d04d7dbbfc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:09.216496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df2d23-2943-41b7-98aa-10ca6eaec9fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-80df2d23-2943-41b7-98aa-10ca6eaec9fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:09.221811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:09.222700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:09.223512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 129/274] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:09.228631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df2d23-2943-41b7-98aa-10ca6eaec9fa None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:09.276298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:09.321649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f53e8b-d1ca-430a-8cdf-36d04d7dbbfc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:09.339560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df2d23-2943-41b7-98aa-10ca6eaec9fa None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:09.361466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf08c06-7f9b-478b-b215-92b1dc1eaa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:09.362402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf08c06-7f9b-478b-b215-92b1dc1eaa1d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:09.362700 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf08c06-7f9b-478b-b215-92b1dc1eaa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:09.362976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf08c06-7f9b-478b-b215-92b1dc1eaa1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:09.363337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:09.365491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf08c06-7f9b-478b-b215-92b1dc1eaa1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:09.367828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf08c06-7f9b-478b-b215-92b1dc1eaa1d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:09.370064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 146/275] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:09.383121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf08c06-7f9b-478b-b215-92b1dc1eaa1d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:09.383317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649bad2-f477-4f2e-9632-5f1357e1467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:09.383317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649bad2-f477-4f2e-9632-5f1357e1467c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:09.383317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649bad2-f477-4f2e-9632-5f1357e1467c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:09.383317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649bad2-f477-4f2e-9632-5f1357e1467c None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:09.383784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649bad2-f477-4f2e-9632-5f1357e1467c None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:09.384128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649bad2-f477-4f2e-9632-5f1357e1467c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:09.385661 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649bad2-f477-4f2e-9632-5f1357e1467c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:09.401635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 130/276] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:09.402740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649bad2-f477-4f2e-9632-5f1357e1467c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:09.493352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf08c06-7f9b-478b-b215-92b1dc1eaa1d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:09.524878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:09.532021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e364db-9d54-4c86-8531-21fe02c84ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:09.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e364db-9d54-4c86-8531-21fe02c84ba6 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:09.532981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e364db-9d54-4c86-8531-21fe02c84ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:09.533263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e364db-9d54-4c86-8531-21fe02c84ba6 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:09.534207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649bad2-f477-4f2e-9632-5f1357e1467c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:09.534303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e364db-9d54-4c86-8531-21fe02c84ba6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:09.535496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e364db-9d54-4c86-8531-21fe02c84ba6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:09.569509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6badf74-eabd-4de9-ae50-d3eab6174f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:09.570144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6badf74-eabd-4de9-ae50-d3eab6174f2a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:09.570427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6badf74-eabd-4de9-ae50-d3eab6174f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:09.570731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6badf74-eabd-4de9-ae50-d3eab6174f2a None admin] PATH_INFO: `/v3/users/abeb745ae5e843edba6f38d634ad45c0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:09.583988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6badf74-eabd-4de9-ae50-d3eab6174f2a None admin] RBAC: Authorizing `identity:get_user(user_id=abeb745ae5e843edba6f38d634ad45c0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:09.585487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6badf74-eabd-4de9-ae50-d3eab6174f2a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:09.587212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 131/277] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:09 2022] GET /identity/v3/users/abeb745ae5e843edba6f38d634ad45c0 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:09.592015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161290a1-def8-4569-b9f4-b28b36fee161 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:09.592015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161290a1-def8-4569-b9f4-b28b36fee161 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:09.592015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161290a1-def8-4569-b9f4-b28b36fee161 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:09.592015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 132/278] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:09.592015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed0937-d0c3-4a94-a747-9b25f92d7548 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:09.592015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed0937-d0c3-4a94-a747-9b25f92d7548 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:09.592388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed0937-d0c3-4a94-a747-9b25f92d7548 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:09.593165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 133/279] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:09.593858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed0937-d0c3-4a94-a747-9b25f92d7548 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:09.629440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 147/280] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:46:09 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:46:09.631282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de211614-b53c-4a3a-bb83-afd094a765d9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:09.631567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de211614-b53c-4a3a-bb83-afd094a765d9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:09.631838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de211614-b53c-4a3a-bb83-afd094a765d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:09.639106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:09.651237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-de211614-b53c-4a3a-bb83-afd094a765d9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:09.652608 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-de211614-b53c-4a3a-bb83-afd094a765d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:09.708980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed0937-d0c3-4a94-a747-9b25f92d7548 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:09.734836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 148/281] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:09.736160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79d50b-3795-4b51-9190-778aa98ba7dc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:09.736452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79d50b-3795-4b51-9190-778aa98ba7dc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:09.736730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79d50b-3795-4b51-9190-778aa98ba7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:09.758587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5b79d50b-3795-4b51-9190-778aa98ba7dc None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:09.759851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5b79d50b-3795-4b51-9190-778aa98ba7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:09.767854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec49896-161d-42fd-aafb-635420a6f613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:09.768801 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec49896-161d-42fd-aafb-635420a6f613 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:09.769245 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec49896-161d-42fd-aafb-635420a6f613 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:09.770242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec49896-161d-42fd-aafb-635420a6f613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:09.771204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec49896-161d-42fd-aafb-635420a6f613 None admin] RBAC: Adding query filter params (role.id=991609c9013b45b3907788ef9b42d051, user.id=abeb745ae5e843edba6f38d634ad45c0, scope.system=all) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:09.771638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec49896-161d-42fd-aafb-635420a6f613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:09.774315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec49896-161d-42fd-aafb-635420a6f613 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:09.787214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 134/282] 10.222.0.59 () {56 vars in 1308 bytes} [Fri Sep 16 07:46:09 2022] GET /identity/v3/role_assignments?user.id=abeb745ae5e843edba6f38d634ad45c0&scope.system=all&role.id=991609c9013b45b3907788ef9b42d051 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:09.850071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 149/283] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:10.015718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec49896-161d-42fd-aafb-635420a6f613 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:10.133711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79d50b-3795-4b51-9190-778aa98ba7dc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:10.134117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec49896-161d-42fd-aafb-635420a6f613 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:10.142452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:10.186472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef1b48-4014-4c06-a225-10c73462a494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:10.187183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef1b48-4014-4c06-a225-10c73462a494 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:10.187465 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef1b48-4014-4c06-a225-10c73462a494 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:10.187740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef1b48-4014-4c06-a225-10c73462a494 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:10.207107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef1b48-4014-4c06-a225-10c73462a494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-49ef1b48-4014-4c06-a225-10c73462a494 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:10.213039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:10.214067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:10.215441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 135/284] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:10.217573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:10.222265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef1b48-4014-4c06-a225-10c73462a494 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:10.295769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79d50b-3795-4b51-9190-778aa98ba7dc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:10.346345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf857652-718b-4b74-ba29-251a52bcce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:10.347019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf857652-718b-4b74-ba29-251a52bcce16 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:10.347296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf857652-718b-4b74-ba29-251a52bcce16 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:10.347565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf857652-718b-4b74-ba29-251a52bcce16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:10.370644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf857652-718b-4b74-ba29-251a52bcce16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-bf857652-718b-4b74-ba29-251a52bcce16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:10.375562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:10.376440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:10.377223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 150/285] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:10.384398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf857652-718b-4b74-ba29-251a52bcce16 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:10.405732 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef1b48-4014-4c06-a225-10c73462a494 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:10.467971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0344b6-175e-491d-9b7f-3722734a059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:10.468599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0344b6-175e-491d-9b7f-3722734a059f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:10.468882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0344b6-175e-491d-9b7f-3722734a059f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:10.469159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0344b6-175e-491d-9b7f-3722734a059f None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:10.469835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0344b6-175e-491d-9b7f-3722734a059f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:10.470141 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0344b6-175e-491d-9b7f-3722734a059f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:10.471592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0344b6-175e-491d-9b7f-3722734a059f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:10.495022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 136/286] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:10.496182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0344b6-175e-491d-9b7f-3722734a059f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:10.510692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf857652-718b-4b74-ba29-251a52bcce16 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:10.518836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:10.547028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f349b2c-a0bc-4e2e-bb4d-f82d87f6e28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:10.547502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f349b2c-a0bc-4e2e-bb4d-f82d87f6e28e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:10.547794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f349b2c-a0bc-4e2e-bb4d-f82d87f6e28e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:10.548172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f349b2c-a0bc-4e2e-bb4d-f82d87f6e28e None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:10.548336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:10.549862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f349b2c-a0bc-4e2e-bb4d-f82d87f6e28e None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:10.549862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f349b2c-a0bc-4e2e-bb4d-f82d87f6e28e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:10.551283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f349b2c-a0bc-4e2e-bb4d-f82d87f6e28e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:10.560362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 151/287] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:10.571718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f349b2c-a0bc-4e2e-bb4d-f82d87f6e28e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:10.658157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0344b6-175e-491d-9b7f-3722734a059f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:10.723364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd8e9d-9f30-4153-bcbb-14cf2c623bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:10.724008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd8e9d-9f30-4153-bcbb-14cf2c623bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:10.724347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd8e9d-9f30-4153-bcbb-14cf2c623bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:10.724624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd8e9d-9f30-4153-bcbb-14cf2c623bd6 None admin] PATH_INFO: `/v3/users/fdac03db82fc4fbca4211d421e0c5be9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:10.725828 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd8e9d-9f30-4153-bcbb-14cf2c623bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=fdac03db82fc4fbca4211d421e0c5be9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:10.727335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd8e9d-9f30-4153-bcbb-14cf2c623bd6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:10.729355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 137/288] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:10 2022] GET /identity/v3/users/fdac03db82fc4fbca4211d421e0c5be9 => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:10.735336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd8e9d-9f30-4153-bcbb-14cf2c623bd6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:10.766547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f349b2c-a0bc-4e2e-bb4d-f82d87f6e28e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:10.838799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f4a3a-b763-488c-9515-1849d8d0cb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:10.839988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f4a3a-b763-488c-9515-1849d8d0cb61 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:10.840323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f4a3a-b763-488c-9515-1849d8d0cb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:10.840601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f4a3a-b763-488c-9515-1849d8d0cb61 None admin] PATH_INFO: `/v3/users/43f74119163e4eabb8e6bde3a0ebadf2` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:10.843131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f4a3a-b763-488c-9515-1849d8d0cb61 None admin] RBAC: Authorizing `identity:get_user(user_id=43f74119163e4eabb8e6bde3a0ebadf2)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:10.845984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f4a3a-b763-488c-9515-1849d8d0cb61 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:10.847679 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 152/289] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:10 2022] GET /identity/v3/users/43f74119163e4eabb8e6bde3a0ebadf2 => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:10.853593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f4a3a-b763-488c-9515-1849d8d0cb61 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:10.915034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:10.917360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:10.925119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd8e9d-9f30-4153-bcbb-14cf2c623bd6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:10.990188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432102b-d5c9-4a19-8d8c-92a1f23c6a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:10.990875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432102b-d5c9-4a19-8d8c-92a1f23c6a22 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:10.991212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432102b-d5c9-4a19-8d8c-92a1f23c6a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:10.991497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432102b-d5c9-4a19-8d8c-92a1f23c6a22 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:10.992646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432102b-d5c9-4a19-8d8c-92a1f23c6a22 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:10.994112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432102b-d5c9-4a19-8d8c-92a1f23c6a22 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:10.995817 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 138/290] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:10 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:11.001500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432102b-d5c9-4a19-8d8c-92a1f23c6a22 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:11.042524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f4a3a-b763-488c-9515-1849d8d0cb61 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:11.108743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c4299-79c6-4e5f-96db-c666ea5fc110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:11.109277 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c4299-79c6-4e5f-96db-c666ea5fc110 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:11.109555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c4299-79c6-4e5f-96db-c666ea5fc110 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:11.109828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c4299-79c6-4e5f-96db-c666ea5fc110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:11.184451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432102b-d5c9-4a19-8d8c-92a1f23c6a22 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:11.228011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c4299-79c6-4e5f-96db-c666ea5fc110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:11.229490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c4299-79c6-4e5f-96db-c666ea5fc110 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:11.260317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c565ab-2c39-486b-8e91-b574bbd63b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:11.260993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c565ab-2c39-486b-8e91-b574bbd63b8b None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:11.261799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c565ab-2c39-486b-8e91-b574bbd63b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:11.262077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c565ab-2c39-486b-8e91-b574bbd63b8b None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/users/fdac03db82fc4fbca4211d421e0c5be9/roles/b6562af5bda44cc18b203749c9cc1ef3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:11.264108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c565ab-2c39-486b-8e91-b574bbd63b8b None admin] RBAC: Authorizing `identity:create_grant(project_id=d64c95c677b44381b0ee5c5ab3c1d15d, user_id=fdac03db82fc4fbca4211d421e0c5be9, role_id=b6562af5bda44cc18b203749c9cc1ef3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:11.264880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:11.265683 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c565ab-2c39-486b-8e91-b574bbd63b8b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:11.333086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 153/291] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:11.340306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c4299-79c6-4e5f-96db-c666ea5fc110 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:11.377088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 139/292] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:46:11 2022] PUT /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/users/fdac03db82fc4fbca4211d421e0c5be9/roles/b6562af5bda44cc18b203749c9cc1ef3 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 07:46:11.467510 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c4299-79c6-4e5f-96db-c666ea5fc110 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:11.521248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80edc9c-98fa-4251-96c0-999971626182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:11.521906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80edc9c-98fa-4251-96c0-999971626182 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:11.522187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80edc9c-98fa-4251-96c0-999971626182 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:11.522464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80edc9c-98fa-4251-96c0-999971626182 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:11.527560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80edc9c-98fa-4251-96c0-999971626182 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:11.529056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80edc9c-98fa-4251-96c0-999971626182 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:11.530995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 154/293] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:11 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:11.537360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c565ab-2c39-486b-8e91-b574bbd63b8b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:11.587575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:11.675266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c565ab-2c39-486b-8e91-b574bbd63b8b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:11.712190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf90b1e-5890-42ea-a22c-14e3e2a2832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:11.713157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf90b1e-5890-42ea-a22c-14e3e2a2832b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:11.713630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf90b1e-5890-42ea-a22c-14e3e2a2832b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:11.714094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf90b1e-5890-42ea-a22c-14e3e2a2832b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:11.715090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf90b1e-5890-42ea-a22c-14e3e2a2832b None admin] RBAC: Adding query filter params (role.id=991609c9013b45b3907788ef9b42d051, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d, user.id=43f74119163e4eabb8e6bde3a0ebadf2) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:11.715613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf90b1e-5890-42ea-a22c-14e3e2a2832b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:11.718312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf90b1e-5890-42ea-a22c-14e3e2a2832b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:11.725126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 140/294] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:11 2022] GET /identity/v3/role_assignments?user.id=43f74119163e4eabb8e6bde3a0ebadf2&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=991609c9013b45b3907788ef9b42d051 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:12.235587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d524658-185e-4a7b-b65b-19223f3c1d97 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:12.236264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d524658-185e-4a7b-b65b-19223f3c1d97 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:12.236768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d524658-185e-4a7b-b65b-19223f3c1d97 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:12.238549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 155/295] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:12.244272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ea260-1ee4-478f-bb65-83efacf3b488 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:12.244802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ea260-1ee4-478f-bb65-83efacf3b488 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:12.245285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ea260-1ee4-478f-bb65-83efacf3b488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:12.265313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a45ea260-1ee4-478f-bb65-83efacf3b488 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:12.266871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a45ea260-1ee4-478f-bb65-83efacf3b488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:12.334960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9992b-c15a-4fcb-8c4c-98e421ecf592 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:12.335932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9992b-c15a-4fcb-8c4c-98e421ecf592 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:12.336415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9992b-c15a-4fcb-8c4c-98e421ecf592 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:12.338061 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 156/296] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:46:12.353681 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 141/297] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:12.359543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25783a-fa05-4252-8ac4-319894b64603 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:12.360325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25783a-fa05-4252-8ac4-319894b64603 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:12.360905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25783a-fa05-4252-8ac4-319894b64603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:12.368154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:12.379594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ae25783a-fa05-4252-8ac4-319894b64603 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:12.380845 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ae25783a-fa05-4252-8ac4-319894b64603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:12.432248 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e106cb-bf7d-4e63-9494-2e53796b276f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:12.432572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e106cb-bf7d-4e63-9494-2e53796b276f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:12.432799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e106cb-bf7d-4e63-9494-2e53796b276f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:12.434970 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 142/298] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:46:12.440280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a318c7-d46a-465f-a28c-d1ef11bf1dae None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:12.440563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a318c7-d46a-465f-a28c-d1ef11bf1dae None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:12.440840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a318c7-d46a-465f-a28c-d1ef11bf1dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:12.452643 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 157/299] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:12.459459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c0a318c7-d46a-465f-a28c-d1ef11bf1dae None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:12.460829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c0a318c7-d46a-465f-a28c-d1ef11bf1dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:12.474569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:12.538797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 143/300] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:46:12.673472 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25783a-fa05-4252-8ac4-319894b64603 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:12.735462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a318c7-d46a-465f-a28c-d1ef11bf1dae None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:12.794791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25783a-fa05-4252-8ac4-319894b64603 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:12.836271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3717e5-f162-415e-b5b9-6418bd2f87b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:12.837062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3717e5-f162-415e-b5b9-6418bd2f87b3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:12.837723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3717e5-f162-415e-b5b9-6418bd2f87b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:12.838004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3717e5-f162-415e-b5b9-6418bd2f87b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:12.840418 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:12.849069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3717e5-f162-415e-b5b9-6418bd2f87b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-3b3717e5-f162-415e-b5b9-6418bd2f87b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:12.853634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:12.854500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:12.855357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 158/301] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:12.856623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3717e5-f162-415e-b5b9-6418bd2f87b3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:12.863524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a318c7-d46a-465f-a28c-d1ef11bf1dae None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:12.904933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48961b43-b019-4d7c-a829-10174e864bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:12.905925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961b43-b019-4d7c-a829-10174e864bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:12.906214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961b43-b019-4d7c-a829-10174e864bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:12.906624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961b43-b019-4d7c-a829-10174e864bf3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:12.917341 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48961b43-b019-4d7c-a829-10174e864bf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-48961b43-b019-4d7c-a829-10174e864bf3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:12.922251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:12.923214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:12.924009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 144/302] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:12.924671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48961b43-b019-4d7c-a829-10174e864bf3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:12.940327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:12.965787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3717e5-f162-415e-b5b9-6418bd2f87b3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:13.001063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef92e66-bea8-4a49-86e5-14398b2b12e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:13.001707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef92e66-bea8-4a49-86e5-14398b2b12e6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:13.001988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef92e66-bea8-4a49-86e5-14398b2b12e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:13.002262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef92e66-bea8-4a49-86e5-14398b2b12e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:13.003407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef92e66-bea8-4a49-86e5-14398b2b12e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:13.004849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef92e66-bea8-4a49-86e5-14398b2b12e6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:13.006594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 159/303] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:13.007317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef92e66-bea8-4a49-86e5-14398b2b12e6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:13.039752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48961b43-b019-4d7c-a829-10174e864bf3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:13.072335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:13.078251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d012c4-98b9-450b-84ca-5accfe7da5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:13.078887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d012c4-98b9-450b-84ca-5accfe7da5ab None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:13.079173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d012c4-98b9-450b-84ca-5accfe7da5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:13.079515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d012c4-98b9-450b-84ca-5accfe7da5ab None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:13.080160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d012c4-98b9-450b-84ca-5accfe7da5ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:13.080459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d012c4-98b9-450b-84ca-5accfe7da5ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:13.082029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d012c4-98b9-450b-84ca-5accfe7da5ab None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:13.095629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 145/304] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:13.096342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d012c4-98b9-450b-84ca-5accfe7da5ab None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:13.114331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef92e66-bea8-4a49-86e5-14398b2b12e6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:13.149509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d468894-a6a2-41e3-ba25-d2b42fac1d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:13.150116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d468894-a6a2-41e3-ba25-d2b42fac1d01 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:13.150398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d468894-a6a2-41e3-ba25-d2b42fac1d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:13.150694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d468894-a6a2-41e3-ba25-d2b42fac1d01 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:13.151462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d468894-a6a2-41e3-ba25-d2b42fac1d01 None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:13.151662 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d468894-a6a2-41e3-ba25-d2b42fac1d01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:13.153574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d468894-a6a2-41e3-ba25-d2b42fac1d01 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:13.162063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 160/305] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:13.162782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d468894-a6a2-41e3-ba25-d2b42fac1d01 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:13.168364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:13.206697 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d012c4-98b9-450b-84ca-5accfe7da5ab None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:13.239612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bf719-9647-4584-b3eb-2f70edfb3c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:13.240227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf719-9647-4584-b3eb-2f70edfb3c5e None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:13.240512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf719-9647-4584-b3eb-2f70edfb3c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:13.240783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf719-9647-4584-b3eb-2f70edfb3c5e None admin] PATH_INFO: `/v3/groups` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:13.241413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bf719-9647-4584-b3eb-2f70edfb3c5e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:13.242882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bf719-9647-4584-b3eb-2f70edfb3c5e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:13.265030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d468894-a6a2-41e3-ba25-d2b42fac1d01 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:13.287700 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:13.298600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963ef5-cd8f-4f49-859b-83004b8d7cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:13.300743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963ef5-cd8f-4f49-859b-83004b8d7cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:13.301145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963ef5-cd8f-4f49-859b-83004b8d7cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:13.301424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963ef5-cd8f-4f49-859b-83004b8d7cf8 None admin] PATH_INFO: `/v3/users/ae85e4e5ab8b4f718cb1e01f08ea6bbd` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:13.308669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 146/306] 10.222.0.59 () {60 vars in 1134 bytes} [Fri Sep 16 07:46:13 2022] POST /identity/v3/groups => generated 229 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:46:13.315912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52963ef5-cd8f-4f49-859b-83004b8d7cf8 None admin] RBAC: Authorizing `identity:get_user(user_id=ae85e4e5ab8b4f718cb1e01f08ea6bbd)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:13.317400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52963ef5-cd8f-4f49-859b-83004b8d7cf8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:13.319430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 161/307] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:13 2022] GET /identity/v3/users/ae85e4e5ab8b4f718cb1e01f08ea6bbd => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:13.319919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bf719-9647-4584-b3eb-2f70edfb3c5e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:13.325892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963ef5-cd8f-4f49-859b-83004b8d7cf8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:13.464182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52963ef5-cd8f-4f49-859b-83004b8d7cf8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:13.474746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bf719-9647-4584-b3eb-2f70edfb3c5e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:13.512928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db4024-7432-4558-96aa-402ef50915a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:13.514238 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db4024-7432-4558-96aa-402ef50915a3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:13.514238 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db4024-7432-4558-96aa-402ef50915a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:13.514238 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db4024-7432-4558-96aa-402ef50915a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:13.514835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db4024-7432-4558-96aa-402ef50915a3 None admin] RBAC: Adding query filter params (role.id=b6562af5bda44cc18b203749c9cc1ef3, user.id=ae85e4e5ab8b4f718cb1e01f08ea6bbd, scope.system=all) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:13.515144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db4024-7432-4558-96aa-402ef50915a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:13.516494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:13.516606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db4024-7432-4558-96aa-402ef50915a3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:13.521692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 162/308] 10.222.0.59 () {56 vars in 1308 bytes} [Fri Sep 16 07:46:13 2022] GET /identity/v3/role_assignments?user.id=ae85e4e5ab8b4f718cb1e01f08ea6bbd&scope.system=all&role.id=b6562af5bda44cc18b203749c9cc1ef3 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:13.527679 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5575e-6d74-4dbe-bd60-32636c6a7f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:13.528438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5575e-6d74-4dbe-bd60-32636c6a7f6c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:13.532535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5575e-6d74-4dbe-bd60-32636c6a7f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:13.532535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5575e-6d74-4dbe-bd60-32636c6a7f6c None admin] PATH_INFO: `/v3/users/abeb745ae5e843edba6f38d634ad45c0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:13.532535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5575e-6d74-4dbe-bd60-32636c6a7f6c None admin] RBAC: Authorizing `identity:get_user(user_id=abeb745ae5e843edba6f38d634ad45c0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:13.532535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5575e-6d74-4dbe-bd60-32636c6a7f6c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:13.535533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 147/309] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:13 2022] GET /identity/v3/users/abeb745ae5e843edba6f38d634ad45c0 => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:13.541879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db4024-7432-4558-96aa-402ef50915a3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:13.611799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:13.674702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db4024-7432-4558-96aa-402ef50915a3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:13.724842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ba0bc-c082-4bac-8d35-cf181d155d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:13.724842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba0bc-c082-4bac-8d35-cf181d155d61 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:13.724842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba0bc-c082-4bac-8d35-cf181d155d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:13.724842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba0bc-c082-4bac-8d35-cf181d155d61 None admin] PATH_INFO: `/v3/system/users/abeb745ae5e843edba6f38d634ad45c0/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:13.725210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ba0bc-c082-4bac-8d35-cf181d155d61 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=abeb745ae5e843edba6f38d634ad45c0, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:13.725210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ba0bc-c082-4bac-8d35-cf181d155d61 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:13.822767 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 163/310] 10.222.0.59 () {58 vars in 1334 bytes} [Fri Sep 16 07:46:13 2022] PUT /identity/v3/system/users/abeb745ae5e843edba6f38d634ad45c0/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 07:46:13.977349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63e73a-12c8-41ce-8958-2806c35bab1d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:13.977905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63e73a-12c8-41ce-8958-2806c35bab1d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:13.977905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63e73a-12c8-41ce-8958-2806c35bab1d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:13.979380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 148/311] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:13.985363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be14e8e-3204-4642-9595-9bb780f9bf54 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:13.985715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be14e8e-3204-4642-9595-9bb780f9bf54 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:13.985920 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be14e8e-3204-4642-9595-9bb780f9bf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:14.006496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9be14e8e-3204-4642-9595-9bb780f9bf54 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:14.007746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9be14e8e-3204-4642-9595-9bb780f9bf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:14.076213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 164/312] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:46:14.353726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63e73a-12c8-41ce-8958-2806c35bab1d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:14.462801 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63e73a-12c8-41ce-8958-2806c35bab1d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:14.499408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daa609-84e5-468c-8d2f-462cbb23aa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:14.500049 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daa609-84e5-468c-8d2f-462cbb23aa5e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:14.500335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daa609-84e5-468c-8d2f-462cbb23aa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:14.500613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daa609-84e5-468c-8d2f-462cbb23aa5e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:14.510969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daa609-84e5-468c-8d2f-462cbb23aa5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a0daa609-84e5-468c-8d2f-462cbb23aa5e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:14.516330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:14.517496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:14.518318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 149/313] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:14.525760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be14e8e-3204-4642-9595-9bb780f9bf54 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:14.570081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:14.629049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be14e8e-3204-4642-9595-9bb780f9bf54 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:14.664535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b48487-3b34-44b6-ba6b-12a8f7562bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:14.665141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b48487-3b34-44b6-ba6b-12a8f7562bfa None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:14.665416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b48487-3b34-44b6-ba6b-12a8f7562bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:14.665684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b48487-3b34-44b6-ba6b-12a8f7562bfa None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:14.666340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b48487-3b34-44b6-ba6b-12a8f7562bfa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:14.666702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b48487-3b34-44b6-ba6b-12a8f7562bfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:14.668146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b48487-3b34-44b6-ba6b-12a8f7562bfa None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:14.676851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 165/314] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:14.682412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daa609-84e5-468c-8d2f-462cbb23aa5e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:14.748341 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:14.794747 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daa609-84e5-468c-8d2f-462cbb23aa5e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:14.829440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fec113-0145-4c85-b522-2ba92a3364f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:14.830433 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fec113-0145-4c85-b522-2ba92a3364f7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:14.830959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fec113-0145-4c85-b522-2ba92a3364f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:14.831452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fec113-0145-4c85-b522-2ba92a3364f7 None admin] PATH_INFO: `/v3/users/fdac03db82fc4fbca4211d421e0c5be9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:14.832796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fec113-0145-4c85-b522-2ba92a3364f7 None admin] RBAC: Authorizing `identity:get_user(user_id=fdac03db82fc4fbca4211d421e0c5be9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:14.834457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fec113-0145-4c85-b522-2ba92a3364f7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:14.836587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 150/315] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:14 2022] GET /identity/v3/users/fdac03db82fc4fbca4211d421e0c5be9 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:14.842473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b48487-3b34-44b6-ba6b-12a8f7562bfa None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:14.950070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b48487-3b34-44b6-ba6b-12a8f7562bfa None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:14.951576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:14.983735 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8710df-7563-4aef-9d99-62a77d8f639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:14.984345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8710df-7563-4aef-9d99-62a77d8f639d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:14.984619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8710df-7563-4aef-9d99-62a77d8f639d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:14.984889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8710df-7563-4aef-9d99-62a77d8f639d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:15.052697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8710df-7563-4aef-9d99-62a77d8f639d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:15.054198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8710df-7563-4aef-9d99-62a77d8f639d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:15.154173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 166/316] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:15.159745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fec113-0145-4c85-b522-2ba92a3364f7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:15.268110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fec113-0145-4c85-b522-2ba92a3364f7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:15.295428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:15.303937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39b73b-0bfb-4daf-8e86-441e23f9dd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:15.304584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39b73b-0bfb-4daf-8e86-441e23f9dd03 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:15.304863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39b73b-0bfb-4daf-8e86-441e23f9dd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:15.305138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39b73b-0bfb-4daf-8e86-441e23f9dd03 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:15.306283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39b73b-0bfb-4daf-8e86-441e23f9dd03 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:15.307765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39b73b-0bfb-4daf-8e86-441e23f9dd03 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:15.310170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 151/317] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:15 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:15.317618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8710df-7563-4aef-9d99-62a77d8f639d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:15.326996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:15.420707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8710df-7563-4aef-9d99-62a77d8f639d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:15.457012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441026c-4d8f-4c20-9276-d1198f68960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:15.457725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441026c-4d8f-4c20-9276-d1198f68960c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:15.458012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441026c-4d8f-4c20-9276-d1198f68960c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:15.458265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441026c-4d8f-4c20-9276-d1198f68960c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:15.459053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441026c-4d8f-4c20-9276-d1198f68960c None admin] RBAC: Adding query filter params (role.id=b6562af5bda44cc18b203749c9cc1ef3, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d, user.id=fdac03db82fc4fbca4211d421e0c5be9) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:15.459361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441026c-4d8f-4c20-9276-d1198f68960c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:15.460878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441026c-4d8f-4c20-9276-d1198f68960c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:15.468740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 167/318] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:15 2022] GET /identity/v3/role_assignments?user.id=fdac03db82fc4fbca4211d421e0c5be9&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=b6562af5bda44cc18b203749c9cc1ef3 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:15.655872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4510ed-5aff-43d9-afe9-bd47cf076977 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:15.656054 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4510ed-5aff-43d9-afe9-bd47cf076977 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:15.656343 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4510ed-5aff-43d9-afe9-bd47cf076977 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:15.658029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 152/319] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:15.664074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e5794-f0c6-4267-8a08-0093f396d0a2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:15.665028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e5794-f0c6-4267-8a08-0093f396d0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:15.665520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e5794-f0c6-4267-8a08-0093f396d0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:15.687693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-196e5794-f0c6-4267-8a08-0093f396d0a2 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:15.689163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-196e5794-f0c6-4267-8a08-0093f396d0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:15.761823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:15.769656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 168/320] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:15.848272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61763089-6ff2-4066-bcbe-c686db5746b1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:15.848551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61763089-6ff2-4066-bcbe-c686db5746b1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:15.848995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61763089-6ff2-4066-bcbe-c686db5746b1 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:15.850321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 153/321] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:46:15.856006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0aff35-a1e8-422e-9e12-446bb621a978 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:15.856292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0aff35-a1e8-422e-9e12-446bb621a978 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:15.856564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0aff35-a1e8-422e-9e12-446bb621a978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:15.875673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0d0aff35-a1e8-422e-9e12-446bb621a978 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:15.876850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0d0aff35-a1e8-422e-9e12-446bb621a978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:15.946921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 169/322] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:16.041153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61763089-6ff2-4066-bcbe-c686db5746b1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:16.137308 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f41763-bba1-4f37-94d4-1fd352aeeaa1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:16.137596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f41763-bba1-4f37-94d4-1fd352aeeaa1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:16.138101 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f41763-bba1-4f37-94d4-1fd352aeeaa1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:16.139482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 170/323] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:16.144576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a0cce-f1ad-4ffb-bbe3-86981f23b78a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:16.144863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a0cce-f1ad-4ffb-bbe3-86981f23b78a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:16.145250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a0cce-f1ad-4ffb-bbe3-86981f23b78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:16.146561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61763089-6ff2-4066-bcbe-c686db5746b1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:16.164881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-434a0cce-f1ad-4ffb-bbe3-86981f23b78a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:16.166092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-434a0cce-f1ad-4ffb-bbe3-86981f23b78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:16.185534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474f60b-9fb4-41f1-af64-5e495a94ddd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:16.186176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474f60b-9fb4-41f1-af64-5e495a94ddd8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:16.186456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474f60b-9fb4-41f1-af64-5e495a94ddd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:16.186755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474f60b-9fb4-41f1-af64-5e495a94ddd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:16.187874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474f60b-9fb4-41f1-af64-5e495a94ddd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:16.189545 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474f60b-9fb4-41f1-af64-5e495a94ddd8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:16.191302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 154/324] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:16.196453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474f60b-9fb4-41f1-af64-5e495a94ddd8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:16.235885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 171/325] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:46:16.236637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a0cce-f1ad-4ffb-bbe3-86981f23b78a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:16.264337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:16.292588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:16.305603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474f60b-9fb4-41f1-af64-5e495a94ddd8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:16.339834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a0cce-f1ad-4ffb-bbe3-86981f23b78a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:16.340890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74072fcc-80ab-4a8c-9206-3db067ab67ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:16.341551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74072fcc-80ab-4a8c-9206-3db067ab67ba None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:16.341824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74072fcc-80ab-4a8c-9206-3db067ab67ba None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:16.342711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74072fcc-80ab-4a8c-9206-3db067ab67ba None admin] PATH_INFO: `/v3/groups` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:16.342839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74072fcc-80ab-4a8c-9206-3db067ab67ba None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:16.344181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74072fcc-80ab-4a8c-9206-3db067ab67ba None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:16.375115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee17d1-9c42-4a3c-8b02-891362e5e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:16.375752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee17d1-9c42-4a3c-8b02-891362e5e6aa None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:16.376033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee17d1-9c42-4a3c-8b02-891362e5e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:16.376306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee17d1-9c42-4a3c-8b02-891362e5e6aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:16.387031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee17d1-9c42-4a3c-8b02-891362e5e6aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-93ee17d1-9c42-4a3c-8b02-891362e5e6aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:16.392300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:16.393247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:16.394083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 172/326] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:16.399079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee17d1-9c42-4a3c-8b02-891362e5e6aa None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:16.441865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 155/327] 10.222.0.59 () {60 vars in 1134 bytes} [Fri Sep 16 07:46:16 2022] POST /identity/v3/groups => generated 226 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:46:16.508315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee17d1-9c42-4a3c-8b02-891362e5e6aa None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:16.513892 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74072fcc-80ab-4a8c-9206-3db067ab67ba None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:16.528505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:16.547893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73411219-ddc7-4189-ac27-460370fc6256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:16.548521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73411219-ddc7-4189-ac27-460370fc6256 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:16.548793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73411219-ddc7-4189-ac27-460370fc6256 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:16.549062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73411219-ddc7-4189-ac27-460370fc6256 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:16.549742 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73411219-ddc7-4189-ac27-460370fc6256 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:16.550052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73411219-ddc7-4189-ac27-460370fc6256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:16.551672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73411219-ddc7-4189-ac27-460370fc6256 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:16.566581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 173/328] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:16.567962 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73411219-ddc7-4189-ac27-460370fc6256 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:16.641911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74072fcc-80ab-4a8c-9206-3db067ab67ba None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:16.669662 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:16.681607 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4c13c-eb2f-4935-857f-65217018df9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:16.682252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4c13c-eb2f-4935-857f-65217018df9b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:16.682544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4c13c-eb2f-4935-857f-65217018df9b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:16.683355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4c13c-eb2f-4935-857f-65217018df9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:16.685320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73411219-ddc7-4189-ac27-460370fc6256 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:16.697725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4c13c-eb2f-4935-857f-65217018df9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c7b4c13c-eb2f-4935-857f-65217018df9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:16.702759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:16.703857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:16.704936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 156/329] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:16.711292 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4c13c-eb2f-4935-857f-65217018df9b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:16.737821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502be08-a432-453d-b180-343518b0e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:16.738733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502be08-a432-453d-b180-343518b0e6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:16.739154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502be08-a432-453d-b180-343518b0e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:16.739564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502be08-a432-453d-b180-343518b0e6b2 None admin] PATH_INFO: `/v3/users/ae85e4e5ab8b4f718cb1e01f08ea6bbd` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:16.740906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502be08-a432-453d-b180-343518b0e6b2 None admin] RBAC: Authorizing `identity:get_user(user_id=ae85e4e5ab8b4f718cb1e01f08ea6bbd)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:16.742336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502be08-a432-453d-b180-343518b0e6b2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:16.744319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 174/330] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:16 2022] GET /identity/v3/users/ae85e4e5ab8b4f718cb1e01f08ea6bbd => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:16.751197 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502be08-a432-453d-b180-343518b0e6b2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:16.813973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:16.850887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4c13c-eb2f-4935-857f-65217018df9b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:16.882957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502be08-a432-453d-b180-343518b0e6b2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:16.896721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab60963-0e35-438b-8112-b16fac9dd68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:16.897670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab60963-0e35-438b-8112-b16fac9dd68a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:16.898174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab60963-0e35-438b-8112-b16fac9dd68a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:16.898645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab60963-0e35-438b-8112-b16fac9dd68a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:16.899563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab60963-0e35-438b-8112-b16fac9dd68a None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:16.900240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab60963-0e35-438b-8112-b16fac9dd68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:16.901862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab60963-0e35-438b-8112-b16fac9dd68a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:16.911084 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 157/331] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:16.918009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab60963-0e35-438b-8112-b16fac9dd68a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:16.930422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35f614-f97c-43ff-8ee6-7acc62d304e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:16.930892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35f614-f97c-43ff-8ee6-7acc62d304e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:16.931219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35f614-f97c-43ff-8ee6-7acc62d304e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:16.931451 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35f614-f97c-43ff-8ee6-7acc62d304e2 None admin] PATH_INFO: `/v3/system/users/ae85e4e5ab8b4f718cb1e01f08ea6bbd/roles/b6562af5bda44cc18b203749c9cc1ef3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:16.935169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35f614-f97c-43ff-8ee6-7acc62d304e2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ae85e4e5ab8b4f718cb1e01f08ea6bbd, role_id=b6562af5bda44cc18b203749c9cc1ef3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:16.935540 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35f614-f97c-43ff-8ee6-7acc62d304e2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:16.972786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:16.984533 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 175/332] 10.222.0.59 () {58 vars in 1334 bytes} [Fri Sep 16 07:46:16 2022] PUT /identity/v3/system/users/ae85e4e5ab8b4f718cb1e01f08ea6bbd/roles/b6562af5bda44cc18b203749c9cc1ef3 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 07:46:17.037115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab60963-0e35-438b-8112-b16fac9dd68a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:17.079670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a1c1e-4064-499a-b8d2-5978a36b74e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:17.080709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a1c1e-4064-499a-b8d2-5978a36b74e6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:17.081177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a1c1e-4064-499a-b8d2-5978a36b74e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:17.081618 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a1c1e-4064-499a-b8d2-5978a36b74e6 None admin] PATH_INFO: `/v3/users/abeb745ae5e843edba6f38d634ad45c0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:17.083114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a1c1e-4064-499a-b8d2-5978a36b74e6 None admin] RBAC: Authorizing `identity:get_user(user_id=abeb745ae5e843edba6f38d634ad45c0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:17.084922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a1c1e-4064-499a-b8d2-5978a36b74e6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:17.087207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 158/333] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:16 2022] GET /identity/v3/users/abeb745ae5e843edba6f38d634ad45c0 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:17.093253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35f614-f97c-43ff-8ee6-7acc62d304e2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:17.215895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35f614-f97c-43ff-8ee6-7acc62d304e2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:17.227086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:17.253013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af48a9-684e-4891-8296-f8914fc59ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:17.254462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af48a9-684e-4891-8296-f8914fc59ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:17.254462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af48a9-684e-4891-8296-f8914fc59ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:17.254462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af48a9-684e-4891-8296-f8914fc59ad8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:17.255149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af48a9-684e-4891-8296-f8914fc59ad8 None admin] RBAC: Adding query filter params (role.id=991609c9013b45b3907788ef9b42d051, user.id=abeb745ae5e843edba6f38d634ad45c0, scope.system=all) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:17.255462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af48a9-684e-4891-8296-f8914fc59ad8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:17.256921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af48a9-684e-4891-8296-f8914fc59ad8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:17.271437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 176/334] 10.222.0.59 () {56 vars in 1308 bytes} [Fri Sep 16 07:46:17 2022] GET /identity/v3/role_assignments?user.id=abeb745ae5e843edba6f38d634ad45c0&scope.system=all&role.id=991609c9013b45b3907788ef9b42d051 => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:18.853538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc3889-0cd1-428f-844c-d330efed4804 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:18.854603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc3889-0cd1-428f-844c-d330efed4804 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:18.855154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc3889-0cd1-428f-844c-d330efed4804 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:18.856987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 159/335] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:18.862904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57965c6-d9d8-4ef4-87aa-d55c5473a08e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:18.863794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57965c6-d9d8-4ef4-87aa-d55c5473a08e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:18.864286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57965c6-d9d8-4ef4-87aa-d55c5473a08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:18.886375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a57965c6-d9d8-4ef4-87aa-d55c5473a08e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:18.888315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a57965c6-d9d8-4ef4-87aa-d55c5473a08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:19.032069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 177/336] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:46:19.265214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a47dd9-d5e7-4dbe-90a5-647101b4bb25 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:19.265857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a47dd9-d5e7-4dbe-90a5-647101b4bb25 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:19.266322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a47dd9-d5e7-4dbe-90a5-647101b4bb25 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:19.268132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 160/337] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:19.274032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654fbc5-f413-45a1-9954-12451b525702 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:19.274606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654fbc5-f413-45a1-9954-12451b525702 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:19.275133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654fbc5-f413-45a1-9954-12451b525702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:19.299134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f654fbc5-f413-45a1-9954-12451b525702 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:19.300681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f654fbc5-f413-45a1-9954-12451b525702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:19.382834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 178/338] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:19.510090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2aa09a-4957-40e1-bba1-e07ddc3d8928 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:19.510768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2aa09a-4957-40e1-bba1-e07ddc3d8928 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:19.511301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2aa09a-4957-40e1-bba1-e07ddc3d8928 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:19.513158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 161/339] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:19.518031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f498e-4da1-47ca-8649-97e922cc337f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:19.518571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f498e-4da1-47ca-8649-97e922cc337f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:19.519101 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f498e-4da1-47ca-8649-97e922cc337f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:19.543606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-de4f498e-4da1-47ca-8649-97e922cc337f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:19.545106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-de4f498e-4da1-47ca-8649-97e922cc337f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:19.573546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:19.625413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 179/340] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:19.663164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2aa09a-4957-40e1-bba1-e07ddc3d8928 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:19.810568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2aa09a-4957-40e1-bba1-e07ddc3d8928 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:19.826772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:19.832791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f498e-4da1-47ca-8649-97e922cc337f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:19.859662 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5630ae-cd72-49d9-9824-dbd450e63dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:19.860329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5630ae-cd72-49d9-9824-dbd450e63dfa None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:19.860603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5630ae-cd72-49d9-9824-dbd450e63dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:19.860880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5630ae-cd72-49d9-9824-dbd450e63dfa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:19.878736 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5630ae-cd72-49d9-9824-dbd450e63dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5e5630ae-cd72-49d9-9824-dbd450e63dfa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:19.884372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:19.885273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:19.886716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 162/341] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:19.891631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5630ae-cd72-49d9-9824-dbd450e63dfa None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:20.033720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f498e-4da1-47ca-8649-97e922cc337f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:20.067331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5630ae-cd72-49d9-9824-dbd450e63dfa None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:20.092195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1effe-a8ca-43cf-ab76-5294373b8968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:20.092766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1effe-a8ca-43cf-ab76-5294373b8968 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:20.093044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1effe-a8ca-43cf-ab76-5294373b8968 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:20.093315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1effe-a8ca-43cf-ab76-5294373b8968 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:20.108082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1effe-a8ca-43cf-ab76-5294373b8968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-fbb1effe-a8ca-43cf-ab76-5294373b8968 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:20.113334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:20.114214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:20.115106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 180/342] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:46:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:20.122331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1effe-a8ca-43cf-ab76-5294373b8968 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:20.151225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:20.151408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc048e26-875c-4068-a08b-444073841d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:20.152590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc048e26-875c-4068-a08b-444073841d2b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:20.152868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc048e26-875c-4068-a08b-444073841d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:20.153140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc048e26-875c-4068-a08b-444073841d2b None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:20.153817 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc048e26-875c-4068-a08b-444073841d2b None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:20.154119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc048e26-875c-4068-a08b-444073841d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:20.155547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc048e26-875c-4068-a08b-444073841d2b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:20.178945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 163/343] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:20.179800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc048e26-875c-4068-a08b-444073841d2b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:20.255670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:20.302069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1effe-a8ca-43cf-ab76-5294373b8968 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:20.367932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c3e33-973d-4c08-a3f0-a4df0a79de0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:20.368468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c3e33-973d-4c08-a3f0-a4df0a79de0c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:20.368744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c3e33-973d-4c08-a3f0-a4df0a79de0c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:20.369014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c3e33-973d-4c08-a3f0-a4df0a79de0c None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:20.369682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c3e33-973d-4c08-a3f0-a4df0a79de0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:20.369982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c3e33-973d-4c08-a3f0-a4df0a79de0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:20.371418 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c3e33-973d-4c08-a3f0-a4df0a79de0c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:20.380718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc048e26-875c-4068-a08b-444073841d2b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:20.391713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 181/344] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:20.393344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c3e33-973d-4c08-a3f0-a4df0a79de0c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:20.445248 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450dda0-1ce3-461f-bf4c-7de4010bddd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:20.446295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450dda0-1ce3-461f-bf4c-7de4010bddd4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:20.446806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450dda0-1ce3-461f-bf4c-7de4010bddd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:20.447296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450dda0-1ce3-461f-bf4c-7de4010bddd4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:20.476179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450dda0-1ce3-461f-bf4c-7de4010bddd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-e450dda0-1ce3-461f-bf4c-7de4010bddd4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:20.481648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:20.482645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 07:46:20.483513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:20.498246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 164/345] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:20.499515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450dda0-1ce3-461f-bf4c-7de4010bddd4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:20.577548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:20.601447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c3e33-973d-4c08-a3f0-a4df0a79de0c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:20.638150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642e25-df26-458b-9687-4b2a49f7a653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:20.639120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642e25-df26-458b-9687-4b2a49f7a653 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:20.639616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642e25-df26-458b-9687-4b2a49f7a653 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:20.640090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642e25-df26-458b-9687-4b2a49f7a653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:20.678544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:20.685685 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450dda0-1ce3-461f-bf4c-7de4010bddd4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:20.718136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06642e25-df26-458b-9687-4b2a49f7a653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:20.720231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06642e25-df26-458b-9687-4b2a49f7a653 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:20.729504 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1e97d-9910-43ed-9ba3-02db2925a02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:20.730460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e97d-9910-43ed-9ba3-02db2925a02d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:20.730963 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e97d-9910-43ed-9ba3-02db2925a02d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:20.731407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e97d-9910-43ed-9ba3-02db2925a02d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:20.820894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1e97d-9910-43ed-9ba3-02db2925a02d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:20.822715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1e97d-9910-43ed-9ba3-02db2925a02d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:20.827262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:20.850215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 182/346] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:20.851343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642e25-df26-458b-9687-4b2a49f7a653 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:20.956663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:20.965517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 165/347] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:20.967073 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b29bdc-0acf-4798-bef8-a6aa1838c4d9 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:20.967560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b29bdc-0acf-4798-bef8-a6aa1838c4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:20.968044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b29bdc-0acf-4798-bef8-a6aa1838c4d9 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:20.970000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 166/348] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:20.970966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b29bdc-0acf-4798-bef8-a6aa1838c4d9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:20.992694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642e25-df26-458b-9687-4b2a49f7a653 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:21.033984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac82f9-9bea-4f8b-81d6-bd620243ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:21.036388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac82f9-9bea-4f8b-81d6-bd620243ac4d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.036388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac82f9-9bea-4f8b-81d6-bd620243ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.036388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac82f9-9bea-4f8b-81d6-bd620243ac4d None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.036733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac82f9-9bea-4f8b-81d6-bd620243ac4d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:21.037012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac82f9-9bea-4f8b-81d6-bd620243ac4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:21.038388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac82f9-9bea-4f8b-81d6-bd620243ac4d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:21.048135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 183/349] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:21.049467 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ee295-3203-44d4-af2e-b955db6cb298 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.049905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ee295-3203-44d4-af2e-b955db6cb298 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.050328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ee295-3203-44d4-af2e-b955db6cb298 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.051786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 184/350] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:21.056636 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ee295-3203-44d4-af2e-b955db6cb298 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:21.102336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b29bdc-0acf-4798-bef8-a6aa1838c4d9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:21.112850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:21.143540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6af2af-44de-4f70-a509-fc4e1395df44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:21.144453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6af2af-44de-4f70-a509-fc4e1395df44 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.144729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6af2af-44de-4f70-a509-fc4e1395df44 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.145173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6af2af-44de-4f70-a509-fc4e1395df44 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.146466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6af2af-44de-4f70-a509-fc4e1395df44 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:21.148238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6af2af-44de-4f70-a509-fc4e1395df44 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:21.149905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 167/351] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:20 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:21.151194 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47074317-7feb-4b91-b882-6c855e1479c8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.152023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47074317-7feb-4b91-b882-6c855e1479c8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.152301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47074317-7feb-4b91-b882-6c855e1479c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.176351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-47074317-7feb-4b91-b882-6c855e1479c8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:21.177567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-47074317-7feb-4b91-b882-6c855e1479c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:21.192070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ee295-3203-44d4-af2e-b955db6cb298 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:21.240013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b0d44-e610-4175-9a28-780eb3b6682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:21.240566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0d44-e610-4175-9a28-780eb3b6682e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.240851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0d44-e610-4175-9a28-780eb3b6682e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.241116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0d44-e610-4175-9a28-780eb3b6682e None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.242263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b0d44-e610-4175-9a28-780eb3b6682e None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:21.243286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:21.243795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b0d44-e610-4175-9a28-780eb3b6682e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:21.245650 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 185/352] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:21 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:21.246719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b0d44-e610-4175-9a28-780eb3b6682e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:21.263307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 168/353] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:21.264999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7bde-96f4-451f-a3c8-d9ca78134d11 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.265407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7bde-96f4-451f-a3c8-d9ca78134d11 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.265783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7bde-96f4-451f-a3c8-d9ca78134d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.286928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-47be7bde-96f4-451f-a3c8-d9ca78134d11 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:21.288392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-47be7bde-96f4-451f-a3c8-d9ca78134d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:21.365715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b0d44-e610-4175-9a28-780eb3b6682e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:21.373599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 169/354] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:21.374615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be7bde-96f4-451f-a3c8-d9ca78134d11 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:21.380721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:21.412698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d6768-f234-4838-9871-f9d7fdbe2519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:21.414017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d6768-f234-4838-9871-f9d7fdbe2519 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.415869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d6768-f234-4838-9871-f9d7fdbe2519 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.416313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d6768-f234-4838-9871-f9d7fdbe2519 None admin] PATH_INFO: `/v3/users/ae85e4e5ab8b4f718cb1e01f08ea6bbd` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.418295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d6768-f234-4838-9871-f9d7fdbe2519 None admin] RBAC: Authorizing `identity:get_user(user_id=ae85e4e5ab8b4f718cb1e01f08ea6bbd)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:21.419957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d6768-f234-4838-9871-f9d7fdbe2519 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:21.422877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 186/355] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:46:21 2022] GET /identity/v3/users/ae85e4e5ab8b4f718cb1e01f08ea6bbd => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:21.423892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d6768-f234-4838-9871-f9d7fdbe2519 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:21.519162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be7bde-96f4-451f-a3c8-d9ca78134d11 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:21.566175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:21.578423 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d6768-f234-4838-9871-f9d7fdbe2519 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:21.591066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecdc13-22ee-449d-86b1-ebeb248bb410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:21.591725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecdc13-22ee-449d-86b1-ebeb248bb410 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.592003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecdc13-22ee-449d-86b1-ebeb248bb410 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.592274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecdc13-22ee-449d-86b1-ebeb248bb410 None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.603590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecdc13-22ee-449d-86b1-ebeb248bb410 None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:21.605115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecdc13-22ee-449d-86b1-ebeb248bb410 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:21.608873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 170/356] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:21 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:21.609682 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecdc13-22ee-449d-86b1-ebeb248bb410 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:21.650870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f51a79-e6ea-4b60-b3b8-f7b5c6b2bb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:21.652081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f51a79-e6ea-4b60-b3b8-f7b5c6b2bb10 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.652366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f51a79-e6ea-4b60-b3b8-f7b5c6b2bb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.652639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f51a79-e6ea-4b60-b3b8-f7b5c6b2bb10 None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.654460 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f51a79-e6ea-4b60-b3b8-f7b5c6b2bb10 None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:21.655935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f51a79-e6ea-4b60-b3b8-f7b5c6b2bb10 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:21.657525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 187/357] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:21 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:21.658291 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f51a79-e6ea-4b60-b3b8-f7b5c6b2bb10 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:21.725759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:21.757496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecdc13-22ee-449d-86b1-ebeb248bb410 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:21.796576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1233e-bb97-4be0-a846-6a1841d93255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:21.797116 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1233e-bb97-4be0-a846-6a1841d93255 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.797400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1233e-bb97-4be0-a846-6a1841d93255 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.797679 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1233e-bb97-4be0-a846-6a1841d93255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.798418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1233e-bb97-4be0-a846-6a1841d93255 None admin] RBAC: Adding query filter params (role.id=b6562af5bda44cc18b203749c9cc1ef3, user.id=ae85e4e5ab8b4f718cb1e01f08ea6bbd, scope.system=all) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:21.798765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1233e-bb97-4be0-a846-6a1841d93255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:21.798870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f51a79-e6ea-4b60-b3b8-f7b5c6b2bb10 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:21.800250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1233e-bb97-4be0-a846-6a1841d93255 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:21.816980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 171/358] 10.222.0.59 () {56 vars in 1308 bytes} [Fri Sep 16 07:46:21 2022] GET /identity/v3/role_assignments?user.id=ae85e4e5ab8b4f718cb1e01f08ea6bbd&scope.system=all&role.id=b6562af5bda44cc18b203749c9cc1ef3 => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:21.816980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1233e-bb97-4be0-a846-6a1841d93255 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:21.841808 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da649b-7615-4b43-89c6-cc7ac8103001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:21.842441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da649b-7615-4b43-89c6-cc7ac8103001 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.845582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da649b-7615-4b43-89c6-cc7ac8103001 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.845582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da649b-7615-4b43-89c6-cc7ac8103001 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.845582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da649b-7615-4b43-89c6-cc7ac8103001 None admin] RBAC: Adding query filter params (group.id=a6936cdfd246457193c0e4a4c5faca91, role.id=991609c9013b45b3907788ef9b42d051, scope.project.id=1231532f5f8e4acab98360e05e879050) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:21.845582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da649b-7615-4b43-89c6-cc7ac8103001 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:21.845936 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da649b-7615-4b43-89c6-cc7ac8103001 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:21.853554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 188/359] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Sep 16 07:46:21 2022] GET /identity/v3/role_assignments?group.id=a6936cdfd246457193c0e4a4c5faca91&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=991609c9013b45b3907788ef9b42d051 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:21.854233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da649b-7615-4b43-89c6-cc7ac8103001 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:21.883570 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:21.949375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1233e-bb97-4be0-a846-6a1841d93255 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:21.990183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da649b-7615-4b43-89c6-cc7ac8103001 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:21.991134 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4d888-432b-4f60-8db5-479d894b40a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:21.992331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4d888-432b-4f60-8db5-479d894b40a6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:21.992804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4d888-432b-4f60-8db5-479d894b40a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:21.993309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4d888-432b-4f60-8db5-479d894b40a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:21.994278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4d888-432b-4f60-8db5-479d894b40a6 None admin] RBAC: Adding query filter params (group.id=a6936cdfd246457193c0e4a4c5faca91, role.id=64e24908c05145b49cc926880ab769cf, scope.project.id=1231532f5f8e4acab98360e05e879050) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:21.994854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4d888-432b-4f60-8db5-479d894b40a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:21.997153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4d888-432b-4f60-8db5-479d894b40a6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:22.004085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:22.011434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 172/360] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Sep 16 07:46:21 2022] GET /identity/v3/role_assignments?group.id=a6936cdfd246457193c0e4a4c5faca91&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=64e24908c05145b49cc926880ab769cf => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:22.016941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884a8e2-2ccf-488c-b4d0-9dd95664cb3c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:22.016941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884a8e2-2ccf-488c-b4d0-9dd95664cb3c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:22.016941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884a8e2-2ccf-488c-b4d0-9dd95664cb3c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:22.016941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 173/361] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:22.016941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884a8e2-2ccf-488c-b4d0-9dd95664cb3c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:22.039594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1c5d0-5cee-4b6b-a5e4-abdea6eaa07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:22.040928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c5d0-5cee-4b6b-a5e4-abdea6eaa07c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:22.041384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c5d0-5cee-4b6b-a5e4-abdea6eaa07c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:22.041806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c5d0-5cee-4b6b-a5e4-abdea6eaa07c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:22.056892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1c5d0-5cee-4b6b-a5e4-abdea6eaa07c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-0be1c5d0-5cee-4b6b-a5e4-abdea6eaa07c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:22.063839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:22.064948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:22.065773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:22.068761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 189/362] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:22.070387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81625513-957e-4d1b-b72d-bcd871462a29 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:22.070922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81625513-957e-4d1b-b72d-bcd871462a29 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:22.071405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81625513-957e-4d1b-b72d-bcd871462a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:22.093432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-81625513-957e-4d1b-b72d-bcd871462a29 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:22.095088 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-81625513-957e-4d1b-b72d-bcd871462a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:22.181077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884a8e2-2ccf-488c-b4d0-9dd95664cb3c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:22.194004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 190/363] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:22.195677 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81625513-957e-4d1b-b72d-bcd871462a29 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:22.215034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:22.226447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa1834-9692-498d-9537-4ffbfc2ead9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:22.228682 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa1834-9692-498d-9537-4ffbfc2ead9e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:22.228682 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa1834-9692-498d-9537-4ffbfc2ead9e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:22.228682 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa1834-9692-498d-9537-4ffbfc2ead9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:22.242344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa1834-9692-498d-9537-4ffbfc2ead9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8eaa1834-9692-498d-9537-4ffbfc2ead9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:22.248770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:22.249774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:22.250611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:22.253516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 174/364] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:46:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:22.256252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa1834-9692-498d-9537-4ffbfc2ead9e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:22.341810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:22.344549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81625513-957e-4d1b-b72d-bcd871462a29 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:22.394534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa1834-9692-498d-9537-4ffbfc2ead9e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:22.405949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4c71c-d69e-4091-a387-6e2bb4068dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:22.406575 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c71c-d69e-4091-a387-6e2bb4068dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:22.406904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c71c-d69e-4091-a387-6e2bb4068dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:22.407268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c71c-d69e-4091-a387-6e2bb4068dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:22.408029 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4c71c-d69e-4091-a387-6e2bb4068dd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:22.408377 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4c71c-d69e-4091-a387-6e2bb4068dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:22.410410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4c71c-d69e-4091-a387-6e2bb4068dd4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:22.426091 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 191/365] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:22.433487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4c71c-d69e-4091-a387-6e2bb4068dd4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:22.467333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00252bf3-ef9d-435d-b7de-d2d928f3e991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:22.467955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00252bf3-ef9d-435d-b7de-d2d928f3e991 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:22.468236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00252bf3-ef9d-435d-b7de-d2d928f3e991 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:22.470980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00252bf3-ef9d-435d-b7de-d2d928f3e991 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:22.471680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00252bf3-ef9d-435d-b7de-d2d928f3e991 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:22.471997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00252bf3-ef9d-435d-b7de-d2d928f3e991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:22.473403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00252bf3-ef9d-435d-b7de-d2d928f3e991 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:22.496052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 175/366] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:22.499377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00252bf3-ef9d-435d-b7de-d2d928f3e991 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:22.565171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:22.600057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4c71c-d69e-4091-a387-6e2bb4068dd4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:22.636958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a989c-da15-4dce-ba3e-d53b5c0b92d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:22.637933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a989c-da15-4dce-ba3e-d53b5c0b92d9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:22.638434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a989c-da15-4dce-ba3e-d53b5c0b92d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:22.638929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a989c-da15-4dce-ba3e-d53b5c0b92d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:22.650441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00252bf3-ef9d-435d-b7de-d2d928f3e991 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:22.675329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:22.690000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dae7b1-f57a-45ec-9f02-ab018edfe0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:22.690615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dae7b1-f57a-45ec-9f02-ab018edfe0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:22.690914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dae7b1-f57a-45ec-9f02-ab018edfe0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:22.691205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dae7b1-f57a-45ec-9f02-ab018edfe0b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:22.716907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a989c-da15-4dce-ba3e-d53b5c0b92d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:22.719183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a989c-da15-4dce-ba3e-d53b5c0b92d9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:22.773223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dae7b1-f57a-45ec-9f02-ab018edfe0b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:22.775220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dae7b1-f57a-45ec-9f02-ab018edfe0b4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:22.813043 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:22.841362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 192/367] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:22.846295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a989c-da15-4dce-ba3e-d53b5c0b92d9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:22.899209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 176/368] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:22.900503 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dae7b1-f57a-45ec-9f02-ab018edfe0b4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:22.936245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:22.990572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a989c-da15-4dce-ba3e-d53b5c0b92d9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:23.030586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff82d29-c8e5-432e-94cb-a04ca6112b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:23.031284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff82d29-c8e5-432e-94cb-a04ca6112b38 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:23.031390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dae7b1-f57a-45ec-9f02-ab018edfe0b4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:23.031806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff82d29-c8e5-432e-94cb-a04ca6112b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:23.031806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff82d29-c8e5-432e-94cb-a04ca6112b38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:23.047412 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff82d29-c8e5-432e-94cb-a04ca6112b38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-9ff82d29-c8e5-432e-94cb-a04ca6112b38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:23.052406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:23.053345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:23.054133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 193/369] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:46:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:23.054911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff82d29-c8e5-432e-94cb-a04ca6112b38 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:23.073957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:23.085849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f325f-0791-4779-92fb-c48451be5f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:23.086493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f325f-0791-4779-92fb-c48451be5f33 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:23.086810 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f325f-0791-4779-92fb-c48451be5f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:23.087086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f325f-0791-4779-92fb-c48451be5f33 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:23.088237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f325f-0791-4779-92fb-c48451be5f33 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:23.090251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f325f-0791-4779-92fb-c48451be5f33 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:23.092076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 177/370] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:22 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:23.092883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f325f-0791-4779-92fb-c48451be5f33 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:23.199050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff82d29-c8e5-432e-94cb-a04ca6112b38 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:23.208390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f325f-0791-4779-92fb-c48451be5f33 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:23.227715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:23.239712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01610830-deca-43c7-a6da-671fc1380ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:23.240359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610830-deca-43c7-a6da-671fc1380ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:23.240639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610830-deca-43c7-a6da-671fc1380ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:23.240917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610830-deca-43c7-a6da-671fc1380ea8 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:23.242625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01610830-deca-43c7-a6da-671fc1380ea8 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:23.245222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01610830-deca-43c7-a6da-671fc1380ea8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:23.247423 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 194/371] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:23 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:23.248019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01610830-deca-43c7-a6da-671fc1380ea8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:23.254919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70ffe2-0137-483f-a260-157373e60706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:23.255818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ffe2-0137-483f-a260-157373e60706 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:23.256478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ffe2-0137-483f-a260-157373e60706 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:23.256945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ffe2-0137-483f-a260-157373e60706 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:23.258571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70ffe2-0137-483f-a260-157373e60706 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:23.259112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70ffe2-0137-483f-a260-157373e60706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:23.262114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70ffe2-0137-483f-a260-157373e60706 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:23.272879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 178/372] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:23.274347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70ffe2-0137-483f-a260-157373e60706 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:23.354144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:23.374635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01610830-deca-43c7-a6da-671fc1380ea8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:23.410525 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70ffe2-0137-483f-a260-157373e60706 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:23.431973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f867e35-a4f5-4385-9f87-50ba4e012415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:23.433175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f867e35-a4f5-4385-9f87-50ba4e012415 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:23.433459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f867e35-a4f5-4385-9f87-50ba4e012415 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:23.433777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f867e35-a4f5-4385-9f87-50ba4e012415 None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:23.439717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f867e35-a4f5-4385-9f87-50ba4e012415 None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:23.440880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f867e35-a4f5-4385-9f87-50ba4e012415 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:23.442735 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 195/373] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:23 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:23.443416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f867e35-a4f5-4385-9f87-50ba4e012415 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:23.451531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d5eb2-76ed-4921-8bd7-1d6181b33963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:23.454867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d5eb2-76ed-4921-8bd7-1d6181b33963 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:23.455151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d5eb2-76ed-4921-8bd7-1d6181b33963 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:23.455419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d5eb2-76ed-4921-8bd7-1d6181b33963 None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:23.458605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d5eb2-76ed-4921-8bd7-1d6181b33963 None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:23.460127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d5eb2-76ed-4921-8bd7-1d6181b33963 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:23.462033 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 179/374] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:23 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:23.462883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d5eb2-76ed-4921-8bd7-1d6181b33963 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:23.530719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:23.605908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d5eb2-76ed-4921-8bd7-1d6181b33963 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:23.606134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f867e35-a4f5-4385-9f87-50ba4e012415 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:23.640421 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:23.649952 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f429f-b702-4af8-a492-d5089dc2bfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:23.650560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f429f-b702-4af8-a492-d5089dc2bfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:23.650870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f429f-b702-4af8-a492-d5089dc2bfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:23.654858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f429f-b702-4af8-a492-d5089dc2bfb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:23.654858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f429f-b702-4af8-a492-d5089dc2bfb0 None admin] RBAC: Adding query filter params (group.id=a6936cdfd246457193c0e4a4c5faca91, role.id=991609c9013b45b3907788ef9b42d051, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:23.654858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f429f-b702-4af8-a492-d5089dc2bfb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:23.655086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4b097-d0e0-4d3b-b984-f86a17fefb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:23.655086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4b097-d0e0-4d3b-b984-f86a17fefb42 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:23.655086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4b097-d0e0-4d3b-b984-f86a17fefb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:23.655086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4b097-d0e0-4d3b-b984-f86a17fefb42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:23.655346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f429f-b702-4af8-a492-d5089dc2bfb0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:23.660411 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 180/375] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Sep 16 07:46:23 2022] GET /identity/v3/role_assignments?group.id=a6936cdfd246457193c0e4a4c5faca91&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=991609c9013b45b3907788ef9b42d051 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:23.661305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f429f-b702-4af8-a492-d5089dc2bfb0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:23.830500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f429f-b702-4af8-a492-d5089dc2bfb0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:23.836694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4b097-d0e0-4d3b-b984-f86a17fefb42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:23.838361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4b097-d0e0-4d3b-b984-f86a17fefb42 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:23.893044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a6c11-8ce8-465b-a765-89a54f5bd4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:23.893666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a6c11-8ce8-465b-a765-89a54f5bd4fa None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:23.893948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a6c11-8ce8-465b-a765-89a54f5bd4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:23.894226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a6c11-8ce8-465b-a765-89a54f5bd4fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:23.895045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a6c11-8ce8-465b-a765-89a54f5bd4fa None admin] RBAC: Adding query filter params (group.id=a6936cdfd246457193c0e4a4c5faca91, role.id=64e24908c05145b49cc926880ab769cf, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:23.895359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a6c11-8ce8-465b-a765-89a54f5bd4fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:23.896820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a6c11-8ce8-465b-a765-89a54f5bd4fa None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:23.903141 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 181/376] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Sep 16 07:46:23 2022] GET /identity/v3/role_assignments?group.id=a6936cdfd246457193c0e4a4c5faca91&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=64e24908c05145b49cc926880ab769cf => generated 260 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:23.934398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:23.980757 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 196/377] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:23.986268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a6c11-8ce8-465b-a765-89a54f5bd4fa None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:24.048462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:24.110924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a6c11-8ce8-465b-a765-89a54f5bd4fa None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:24.157828 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf3d9c-108e-4ba4-b212-04236296a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:24.158534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3d9c-108e-4ba4-b212-04236296a667 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:24.158855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3d9c-108e-4ba4-b212-04236296a667 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:24.159136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf3d9c-108e-4ba4-b212-04236296a667 None admin] PATH_INFO: `/v3/projects/1dfa255cacf3473b9f6619fec53b8cf1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:24.160338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf3d9c-108e-4ba4-b212-04236296a667 None admin] RBAC: Authorizing `identity:get_project(project_id=1dfa255cacf3473b9f6619fec53b8cf1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:24.161818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf3d9c-108e-4ba4-b212-04236296a667 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:24.163802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 182/378] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:23 2022] GET /identity/v3/projects/1dfa255cacf3473b9f6619fec53b8cf1 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:24.169787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4b097-d0e0-4d3b-b984-f86a17fefb42 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:24.284705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4b097-d0e0-4d3b-b984-f86a17fefb42 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:24.322404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e7864-bfa8-4abc-8168-20b6dee28ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:24.323011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7864-bfa8-4abc-8168-20b6dee28ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:24.323329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7864-bfa8-4abc-8168-20b6dee28ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:24.323603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7864-bfa8-4abc-8168-20b6dee28ac8 None admin] PATH_INFO: `/v3/groups/f4b575263e6e4d779f9cd742523b476e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:24.329760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e7864-bfa8-4abc-8168-20b6dee28ac8 None admin] RBAC: Authorizing `identity:get_group(group_id=f4b575263e6e4d779f9cd742523b476e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:24.331297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e7864-bfa8-4abc-8168-20b6dee28ac8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:24.333248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 197/379] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:24 2022] GET /identity/v3/groups/f4b575263e6e4d779f9cd742523b476e => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:24.339612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf3d9c-108e-4ba4-b212-04236296a667 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:24.462417 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf3d9c-108e-4ba4-b212-04236296a667 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:24.489636 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:24.501196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6985ee-438e-4caf-9aaa-38f03eac7fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:24.501811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6985ee-438e-4caf-9aaa-38f03eac7fad None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:24.502092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6985ee-438e-4caf-9aaa-38f03eac7fad None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:24.502366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6985ee-438e-4caf-9aaa-38f03eac7fad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:24.510884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6985ee-438e-4caf-9aaa-38f03eac7fad None admin] RBAC: Adding query filter params (group.id=f4b575263e6e4d779f9cd742523b476e, role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=1dfa255cacf3473b9f6619fec53b8cf1) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:24.510884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6985ee-438e-4caf-9aaa-38f03eac7fad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:24.510884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6985ee-438e-4caf-9aaa-38f03eac7fad None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:24.510884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 183/380] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Sep 16 07:46:24 2022] GET /identity/v3/role_assignments?group.id=f4b575263e6e4d779f9cd742523b476e&scope.project.id=1dfa255cacf3473b9f6619fec53b8cf1&role.id=62eb3e525077429aacc0ada63300802e => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:24.525251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd16a7-903d-4087-b3bc-39e049b9d934 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:24.525585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd16a7-903d-4087-b3bc-39e049b9d934 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:24.525804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd16a7-903d-4087-b3bc-39e049b9d934 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:24.527816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 198/381] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:24.533920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c93ee6-4504-4a06-a86f-49bcc8538741 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:24.534203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c93ee6-4504-4a06-a86f-49bcc8538741 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:24.534482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c93ee6-4504-4a06-a86f-49bcc8538741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:24.555352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-46c93ee6-4504-4a06-a86f-49bcc8538741 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:24.556593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-46c93ee6-4504-4a06-a86f-49bcc8538741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:24.641677 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 184/382] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:24.875437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adcad1-f1a1-4067-a95a-702905de6470 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:24.875813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adcad1-f1a1-4067-a95a-702905de6470 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:24.876003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adcad1-f1a1-4067-a95a-702905de6470 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:24.877480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 199/383] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:24.891853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02526602-7956-490a-8376-46303d23b4f3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:24.892432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02526602-7956-490a-8376-46303d23b4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:24.892944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02526602-7956-490a-8376-46303d23b4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:24.921256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-02526602-7956-490a-8376-46303d23b4f3 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:24.922822 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-02526602-7956-490a-8376-46303d23b4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:24.952709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adcad1-f1a1-4067-a95a-702905de6470 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:24.994037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:25.041632 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 185/384] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:46:25.108840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adcad1-f1a1-4067-a95a-702905de6470 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:25.148200 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f265c68-75dd-4ce6-a2d7-bacfe3373f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:25.149176 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f265c68-75dd-4ce6-a2d7-bacfe3373f11 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:25.149681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f265c68-75dd-4ce6-a2d7-bacfe3373f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:25.150175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f265c68-75dd-4ce6-a2d7-bacfe3373f11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:25.164917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f265c68-75dd-4ce6-a2d7-bacfe3373f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-1f265c68-75dd-4ce6-a2d7-bacfe3373f11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:25.170853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:25.172160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:25.172992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:25.175683 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 200/385] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:25.182152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02526602-7956-490a-8376-46303d23b4f3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:25.240574 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:25.298234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02526602-7956-490a-8376-46303d23b4f3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:25.334327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691db620-9c5e-410c-842a-9ac9c4fab7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:25.335019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691db620-9c5e-410c-842a-9ac9c4fab7dc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:25.335307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691db620-9c5e-410c-842a-9ac9c4fab7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:25.335581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691db620-9c5e-410c-842a-9ac9c4fab7dc None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:25.336257 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691db620-9c5e-410c-842a-9ac9c4fab7dc None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:25.336563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691db620-9c5e-410c-842a-9ac9c4fab7dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:25.337967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691db620-9c5e-410c-842a-9ac9c4fab7dc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:25.347315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 186/386] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:25.353115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f265c68-75dd-4ce6-a2d7-bacfe3373f11 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:25.449033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:25.468353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f265c68-75dd-4ce6-a2d7-bacfe3373f11 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:25.508910 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619517e6-bdea-49a6-8f32-135369b537df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:25.509609 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619517e6-bdea-49a6-8f32-135369b537df None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:25.509896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619517e6-bdea-49a6-8f32-135369b537df None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:25.510175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619517e6-bdea-49a6-8f32-135369b537df None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:25.511357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619517e6-bdea-49a6-8f32-135369b537df None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:25.512797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619517e6-bdea-49a6-8f32-135369b537df None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:25.514506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 201/387] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:25 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:25.520188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691db620-9c5e-410c-842a-9ac9c4fab7dc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:25.578979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619517e6-bdea-49a6-8f32-135369b537df None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:25.701621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691db620-9c5e-410c-842a-9ac9c4fab7dc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:25.724418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:25.783240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619517e6-bdea-49a6-8f32-135369b537df None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:25.785212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb337b-c4d5-4e29-a8f9-83da86b06493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:25.785856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb337b-c4d5-4e29-a8f9-83da86b06493 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:25.786134 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb337b-c4d5-4e29-a8f9-83da86b06493 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:25.786406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb337b-c4d5-4e29-a8f9-83da86b06493 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:25.788293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb337b-c4d5-4e29-a8f9-83da86b06493 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:25.789746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb337b-c4d5-4e29-a8f9-83da86b06493 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:25.793204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 187/388] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:25 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:25.797565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb337b-c4d5-4e29-a8f9-83da86b06493 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:25.857906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef7fd8-d5e4-4866-ab95-27fc052ad2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:25.858551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef7fd8-d5e4-4866-ab95-27fc052ad2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:25.858859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef7fd8-d5e4-4866-ab95-27fc052ad2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:25.859132 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef7fd8-d5e4-4866-ab95-27fc052ad2d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:25.880919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef7fd8-d5e4-4866-ab95-27fc052ad2d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-a8ef7fd8-d5e4-4866-ab95-27fc052ad2d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:25.887331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:25.888229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:25.889018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:25.891134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 202/389] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:46:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:25.903784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef7fd8-d5e4-4866-ab95-27fc052ad2d2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:25.925685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:26.003142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb337b-c4d5-4e29-a8f9-83da86b06493 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:26.077631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef7fd8-d5e4-4866-ab95-27fc052ad2d2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:26.078651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e1cb4-9538-4671-b08d-8df1ec7ead25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:26.079608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e1cb4-9538-4671-b08d-8df1ec7ead25 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:26.080089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e1cb4-9538-4671-b08d-8df1ec7ead25 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:26.080556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e1cb4-9538-4671-b08d-8df1ec7ead25 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050/groups/a6936cdfd246457193c0e4a4c5faca91/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:26.082819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e1cb4-9538-4671-b08d-8df1ec7ead25 None admin] RBAC: Authorizing `identity:create_grant(project_id=1231532f5f8e4acab98360e05e879050, group_id=a6936cdfd246457193c0e4a4c5faca91, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:26.084486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e1cb4-9538-4671-b08d-8df1ec7ead25 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:26.119723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0a339-20ad-4305-aba1-0f8656b0fdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:26.120376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0a339-20ad-4305-aba1-0f8656b0fdd0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:26.120657 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0a339-20ad-4305-aba1-0f8656b0fdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:26.120931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0a339-20ad-4305-aba1-0f8656b0fdd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:26.121658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0a339-20ad-4305-aba1-0f8656b0fdd0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:26.121965 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0a339-20ad-4305-aba1-0f8656b0fdd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:26.123376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0a339-20ad-4305-aba1-0f8656b0fdd0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:26.132321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 203/390] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:26.139051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0a339-20ad-4305-aba1-0f8656b0fdd0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:26.169357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 188/391] 10.222.0.59 () {58 vars in 1442 bytes} [Fri Sep 16 07:46:25 2022] PUT /identity/v3/projects/1231532f5f8e4acab98360e05e879050/groups/a6936cdfd246457193c0e4a4c5faca91/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:26.222707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:26.254302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0a339-20ad-4305-aba1-0f8656b0fdd0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:26.294904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e13f3-8972-440c-bd08-fec14df7d43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:26.295558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e13f3-8972-440c-bd08-fec14df7d43e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:26.295839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e13f3-8972-440c-bd08-fec14df7d43e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:26.296115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e13f3-8972-440c-bd08-fec14df7d43e None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:26.297246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e13f3-8972-440c-bd08-fec14df7d43e None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:26.298774 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e13f3-8972-440c-bd08-fec14df7d43e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:26.300364 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 204/392] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:26 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:26.306157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e1cb4-9538-4671-b08d-8df1ec7ead25 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:26.348199 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:26.418311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e1cb4-9538-4671-b08d-8df1ec7ead25 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:26.461962 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf73b7-978e-4beb-8612-ea6f45e58748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:26.463613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf73b7-978e-4beb-8612-ea6f45e58748 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:26.463613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf73b7-978e-4beb-8612-ea6f45e58748 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:26.463613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf73b7-978e-4beb-8612-ea6f45e58748 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:26.465324 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf73b7-978e-4beb-8612-ea6f45e58748 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:26.466862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf73b7-978e-4beb-8612-ea6f45e58748 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:26.473718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 189/393] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:26 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:26.475182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e13f3-8972-440c-bd08-fec14df7d43e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:26.585852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e13f3-8972-440c-bd08-fec14df7d43e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:26.623792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04cfa8-e26d-4e1e-b39d-48e261ce9f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:26.624471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04cfa8-e26d-4e1e-b39d-48e261ce9f7c None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:26.624749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04cfa8-e26d-4e1e-b39d-48e261ce9f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:26.625024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04cfa8-e26d-4e1e-b39d-48e261ce9f7c None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050/groups/a6936cdfd246457193c0e4a4c5faca91/roles/64e24908c05145b49cc926880ab769cf` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:26.636228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04cfa8-e26d-4e1e-b39d-48e261ce9f7c None admin] RBAC: Authorizing `identity:create_grant(project_id=1231532f5f8e4acab98360e05e879050, group_id=a6936cdfd246457193c0e4a4c5faca91, role_id=64e24908c05145b49cc926880ab769cf)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:26.637697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04cfa8-e26d-4e1e-b39d-48e261ce9f7c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:26.675315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6d010-786d-4996-8313-1506c017d0b3 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:26.675621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6d010-786d-4996-8313-1506c017d0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:26.675901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6d010-786d-4996-8313-1506c017d0b3 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:26.677577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 190/394] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:46:26.682625 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3662d3-fe48-4002-afd9-7dcac1a2ed51 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:26.682965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3662d3-fe48-4002-afd9-7dcac1a2ed51 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:26.683246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3662d3-fe48-4002-afd9-7dcac1a2ed51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:26.709355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 205/395] 10.222.0.59 () {58 vars in 1442 bytes} [Fri Sep 16 07:46:26 2022] PUT /identity/v3/projects/1231532f5f8e4acab98360e05e879050/groups/a6936cdfd246457193c0e4a4c5faca91/roles/64e24908c05145b49cc926880ab769cf => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:26.709951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:26.715773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ce3662d3-fe48-4002-afd9-7dcac1a2ed51 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:26.717059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ce3662d3-fe48-4002-afd9-7dcac1a2ed51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:26.873676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 191/396] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:27.084295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab73c3-2f7b-4f40-8160-51aaa83ae61f None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:27.084685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab73c3-2f7b-4f40-8160-51aaa83ae61f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:27.084863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab73c3-2f7b-4f40-8160-51aaa83ae61f None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:27.086551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 206/397] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:27.092128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748d53e-6ef8-46cf-8fad-cd0c6435f27b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:27.092414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748d53e-6ef8-46cf-8fad-cd0c6435f27b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:27.092693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748d53e-6ef8-46cf-8fad-cd0c6435f27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:27.123947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3748d53e-6ef8-46cf-8fad-cd0c6435f27b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:27.125100 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3748d53e-6ef8-46cf-8fad-cd0c6435f27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:27.165561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:27.215283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 192/398] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:27.259984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8bf8d-be10-439f-964e-9588a77838ef None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:27.259984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8bf8d-be10-439f-964e-9588a77838ef None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:27.259984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8bf8d-be10-439f-964e-9588a77838ef None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:27.261555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 207/399] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:27.266974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29a288-d3a3-496d-9588-03b442e6c95c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:27.267656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29a288-d3a3-496d-9588-03b442e6c95c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:27.268147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29a288-d3a3-496d-9588-03b442e6c95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:27.295166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0d29a288-d3a3-496d-9588-03b442e6c95c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:27.296755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0d29a288-d3a3-496d-9588-03b442e6c95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:27.296977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8bf8d-be10-439f-964e-9588a77838ef None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:27.409075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 193/400] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:27.462361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8bf8d-be10-439f-964e-9588a77838ef None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:27.508335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deced72f-aa58-46d4-9477-c5a2ecf72eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:27.509496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deced72f-aa58-46d4-9477-c5a2ecf72eec None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:27.509972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deced72f-aa58-46d4-9477-c5a2ecf72eec None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:27.510438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deced72f-aa58-46d4-9477-c5a2ecf72eec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:27.525942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deced72f-aa58-46d4-9477-c5a2ecf72eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-deced72f-aa58-46d4-9477-c5a2ecf72eec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:27.531810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:27.532693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:27.533520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:27.536684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29a288-d3a3-496d-9588-03b442e6c95c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:27.537973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 208/401] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:27.547146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deced72f-aa58-46d4-9477-c5a2ecf72eec None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:27.655218 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:27.704660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:27.709584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deced72f-aa58-46d4-9477-c5a2ecf72eec None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:27.712073 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29a288-d3a3-496d-9588-03b442e6c95c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:27.750023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83de2ed-e830-4950-8a0c-0c45ee03042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:27.751834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83de2ed-e830-4950-8a0c-0c45ee03042b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:27.752107 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83de2ed-e830-4950-8a0c-0c45ee03042b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:27.752475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83de2ed-e830-4950-8a0c-0c45ee03042b None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:27.753375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83de2ed-e830-4950-8a0c-0c45ee03042b None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:27.753534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397722b-62b6-4ef0-a8a5-2f0c7d39c525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:27.753877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83de2ed-e830-4950-8a0c-0c45ee03042b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:27.754471 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397722b-62b6-4ef0-a8a5-2f0c7d39c525 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:27.754919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397722b-62b6-4ef0-a8a5-2f0c7d39c525 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:27.755327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397722b-62b6-4ef0-a8a5-2f0c7d39c525 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:27.755689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83de2ed-e830-4950-8a0c-0c45ee03042b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:27.766723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 209/402] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:27.767682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83de2ed-e830-4950-8a0c-0c45ee03042b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:27.769714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397722b-62b6-4ef0-a8a5-2f0c7d39c525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5397722b-62b6-4ef0-a8a5-2f0c7d39c525 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:27.776118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:27.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:27.778841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:27.780785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 194/403] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:46:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:27.782179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397722b-62b6-4ef0-a8a5-2f0c7d39c525 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:27.882377 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83de2ed-e830-4950-8a0c-0c45ee03042b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:27.893630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397722b-62b6-4ef0-a8a5-2f0c7d39c525 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:27.908668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:27.921426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c169f0e-964e-4fe8-9137-578ad78c5b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:27.922243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c169f0e-964e-4fe8-9137-578ad78c5b3d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:27.922665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c169f0e-964e-4fe8-9137-578ad78c5b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:27.923117 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c169f0e-964e-4fe8-9137-578ad78c5b3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:27.929508 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fd489-832d-462c-bf9c-14bd89edee4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:27.930252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fd489-832d-462c-bf9c-14bd89edee4d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:27.930656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fd489-832d-462c-bf9c-14bd89edee4d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:27.931045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fd489-832d-462c-bf9c-14bd89edee4d None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:27.932218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fd489-832d-462c-bf9c-14bd89edee4d None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:27.936313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fd489-832d-462c-bf9c-14bd89edee4d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:27.936423 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c169f0e-964e-4fe8-9137-578ad78c5b3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:27.936530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 195/404] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:27 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:27.937289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fd489-832d-462c-bf9c-14bd89edee4d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-5c169f0e-964e-4fe8-9137-578ad78c5b3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:27.942246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:27.943182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:46:27.943988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:27.946110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 210/405] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:46:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:27.947059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c169f0e-964e-4fe8-9137-578ad78c5b3d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:27.975384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:28.063430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fd489-832d-462c-bf9c-14bd89edee4d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:28.067513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c169f0e-964e-4fe8-9137-578ad78c5b3d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:28.098137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c9fa2-2c01-41fe-a825-d595e0c4828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:28.098909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c9fa2-2c01-41fe-a825-d595e0c4828d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:28.099306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c9fa2-2c01-41fe-a825-d595e0c4828d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:28.099723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c9fa2-2c01-41fe-a825-d595e0c4828d None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:28.100620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c9fa2-2c01-41fe-a825-d595e0c4828d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:28.100994 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c9fa2-2c01-41fe-a825-d595e0c4828d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:28.102887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c9fa2-2c01-41fe-a825-d595e0c4828d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:28.103464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ba225-9d5c-4184-9710-15eb78aa7b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:28.104409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ba225-9d5c-4184-9710-15eb78aa7b2c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:28.105352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ba225-9d5c-4184-9710-15eb78aa7b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:28.105810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ba225-9d5c-4184-9710-15eb78aa7b2c None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:28.107125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ba225-9d5c-4184-9710-15eb78aa7b2c None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:28.108850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ba225-9d5c-4184-9710-15eb78aa7b2c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:28.110642 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 211/406] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:27 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:28.111583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ba225-9d5c-4184-9710-15eb78aa7b2c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:28.112684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 196/407] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:28.116842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c9fa2-2c01-41fe-a825-d595e0c4828d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:28.163379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:28.205697 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:28.215979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ba225-9d5c-4184-9710-15eb78aa7b2c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:28.221898 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c9fa2-2c01-41fe-a825-d595e0c4828d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:28.255079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec1fdb-2bc1-4e75-8028-ef0c08d6d23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:28.255807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec1fdb-2bc1-4e75-8028-ef0c08d6d23c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:28.256208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec1fdb-2bc1-4e75-8028-ef0c08d6d23c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:28.256546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec1fdb-2bc1-4e75-8028-ef0c08d6d23c None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:28.257298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec1fdb-2bc1-4e75-8028-ef0c08d6d23c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:28.257697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec1fdb-2bc1-4e75-8028-ef0c08d6d23c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:28.259237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec1fdb-2bc1-4e75-8028-ef0c08d6d23c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:28.259828 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0fe00-54bc-41b4-85ec-1ee282359dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:28.260672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0fe00-54bc-41b4-85ec-1ee282359dab None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:28.261092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0fe00-54bc-41b4-85ec-1ee282359dab None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:28.261533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0fe00-54bc-41b4-85ec-1ee282359dab None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/groups/a6936cdfd246457193c0e4a4c5faca91/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:28.263593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0fe00-54bc-41b4-85ec-1ee282359dab None admin] RBAC: Authorizing `identity:create_grant(project_id=d64c95c677b44381b0ee5c5ab3c1d15d, group_id=a6936cdfd246457193c0e4a4c5faca91, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:28.265284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0fe00-54bc-41b4-85ec-1ee282359dab None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:28.272998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 212/408] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:28.272998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec1fdb-2bc1-4e75-8028-ef0c08d6d23c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:28.373957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 197/409] 10.222.0.59 () {58 vars in 1442 bytes} [Fri Sep 16 07:46:28 2022] PUT /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/groups/a6936cdfd246457193c0e4a4c5faca91/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:28.375740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0fe00-54bc-41b4-85ec-1ee282359dab None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:28.378203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec1fdb-2bc1-4e75-8028-ef0c08d6d23c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:28.417160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:28.467877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2b246-c5e0-4fb0-a9f3-6b24ee07255d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:28.468590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2b246-c5e0-4fb0-a9f3-6b24ee07255d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:28.468870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2b246-c5e0-4fb0-a9f3-6b24ee07255d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:28.469140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2b246-c5e0-4fb0-a9f3-6b24ee07255d None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:28.471491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2b246-c5e0-4fb0-a9f3-6b24ee07255d None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:28.472963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2b246-c5e0-4fb0-a9f3-6b24ee07255d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:28.476517 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 213/410] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:28 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:28.482188 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2b246-c5e0-4fb0-a9f3-6b24ee07255d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:28.581582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0fe00-54bc-41b4-85ec-1ee282359dab None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:28.614998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2b246-c5e0-4fb0-a9f3-6b24ee07255d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:28.624567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229dec0-45a9-4b82-b4b4-0140906885e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:28.625566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229dec0-45a9-4b82-b4b4-0140906885e9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:28.625850 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229dec0-45a9-4b82-b4b4-0140906885e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:28.626130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229dec0-45a9-4b82-b4b4-0140906885e9 None admin] PATH_INFO: `/v3/groups/f4b575263e6e4d779f9cd742523b476e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:28.627877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229dec0-45a9-4b82-b4b4-0140906885e9 None admin] RBAC: Authorizing `identity:get_group(group_id=f4b575263e6e4d779f9cd742523b476e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:28.629521 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229dec0-45a9-4b82-b4b4-0140906885e9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:28.632130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 198/411] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:28 2022] GET /identity/v3/groups/f4b575263e6e4d779f9cd742523b476e => generated 229 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:28.637912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229dec0-45a9-4b82-b4b4-0140906885e9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:28.658186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:28.660835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b18af-90ee-408f-a098-e4bc182d3bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:28.661486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b18af-90ee-408f-a098-e4bc182d3bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:28.661763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b18af-90ee-408f-a098-e4bc182d3bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:28.662038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b18af-90ee-408f-a098-e4bc182d3bf4 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:28.664377 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b18af-90ee-408f-a098-e4bc182d3bf4 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:28.665827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b18af-90ee-408f-a098-e4bc182d3bf4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:28.667688 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 214/412] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:28 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:28.675005 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b18af-90ee-408f-a098-e4bc182d3bf4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:28.770730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:28.818859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229dec0-45a9-4b82-b4b4-0140906885e9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:28.826501 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b18af-90ee-408f-a098-e4bc182d3bf4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:28.861712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394ba63-1d4e-461c-9c44-25694c00e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:28.862299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394ba63-1d4e-461c-9c44-25694c00e6b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:28.862582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394ba63-1d4e-461c-9c44-25694c00e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:28.862895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394ba63-1d4e-461c-9c44-25694c00e6b2 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/groups/a6936cdfd246457193c0e4a4c5faca91/roles/64e24908c05145b49cc926880ab769cf` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:28.865835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394ba63-1d4e-461c-9c44-25694c00e6b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d64c95c677b44381b0ee5c5ab3c1d15d, group_id=a6936cdfd246457193c0e4a4c5faca91, role_id=64e24908c05145b49cc926880ab769cf)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:28.867379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394ba63-1d4e-461c-9c44-25694c00e6b2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:28.889295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543f01-278b-4c3c-a358-d2c788384148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:28.889952 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543f01-278b-4c3c-a358-d2c788384148 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:28.890238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543f01-278b-4c3c-a358-d2c788384148 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:28.890572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543f01-278b-4c3c-a358-d2c788384148 None admin] PATH_INFO: `/v3/projects/1dfa255cacf3473b9f6619fec53b8cf1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:28.891849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30543f01-278b-4c3c-a358-d2c788384148 None admin] RBAC: Authorizing `identity:get_project(project_id=1dfa255cacf3473b9f6619fec53b8cf1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:28.893393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30543f01-278b-4c3c-a358-d2c788384148 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:28.895438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 199/413] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:28 2022] GET /identity/v3/projects/1dfa255cacf3473b9f6619fec53b8cf1 => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:28.900917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543f01-278b-4c3c-a358-d2c788384148 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:28.962751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 215/414] 10.222.0.59 () {58 vars in 1442 bytes} [Fri Sep 16 07:46:28 2022] PUT /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d/groups/a6936cdfd246457193c0e4a4c5faca91/roles/64e24908c05145b49cc926880ab769cf => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:29.015605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:29.044765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543f01-278b-4c3c-a358-d2c788384148 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:29.107713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c82e7-da71-455d-b106-af1a3ee5fb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:29.108729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c82e7-da71-455d-b106-af1a3ee5fb89 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:29.109145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c82e7-da71-455d-b106-af1a3ee5fb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:29.109581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c82e7-da71-455d-b106-af1a3ee5fb89 None admin] PATH_INFO: `/v3/projects/1dfa255cacf3473b9f6619fec53b8cf1/groups/f4b575263e6e4d779f9cd742523b476e/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:29.113240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c82e7-da71-455d-b106-af1a3ee5fb89 None admin] RBAC: Authorizing `identity:create_grant(project_id=1dfa255cacf3473b9f6619fec53b8cf1, group_id=f4b575263e6e4d779f9cd742523b476e, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:29.116203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c82e7-da71-455d-b106-af1a3ee5fb89 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:29.217854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 200/415] 10.222.0.59 () {58 vars in 1442 bytes} [Fri Sep 16 07:46:28 2022] PUT /identity/v3/projects/1dfa255cacf3473b9f6619fec53b8cf1/groups/f4b575263e6e4d779f9cd742523b476e/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 07:46:29.241491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c87f2-c3cb-43ef-baca-4781d4ead0be None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:29.242019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c87f2-c3cb-43ef-baca-4781d4ead0be None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:29.242504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c87f2-c3cb-43ef-baca-4781d4ead0be None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:29.245686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 216/416] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:29.251855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0ab33-8407-44e6-9b35-f2b596137ba8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:29.252280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0ab33-8407-44e6-9b35-f2b596137ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:29.252698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0ab33-8407-44e6-9b35-f2b596137ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:29.282954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-08a0ab33-8407-44e6-9b35-f2b596137ba8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:29.284773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-08a0ab33-8407-44e6-9b35-f2b596137ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:29.414323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 201/417] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:46:29.610147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d41006-5a80-408e-b435-49664bdb23bc None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:29.611183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d41006-5a80-408e-b435-49664bdb23bc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:29.611675 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d41006-5a80-408e-b435-49664bdb23bc None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:29.613477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 217/418] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:29.619588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8482db-af17-4a06-96c8-c5c476cbbdb0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:29.620479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8482db-af17-4a06-96c8-c5c476cbbdb0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:29.620961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8482db-af17-4a06-96c8-c5c476cbbdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:29.639201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6d8482db-af17-4a06-96c8-c5c476cbbdb0 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:29.642088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6d8482db-af17-4a06-96c8-c5c476cbbdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:29.733997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d41006-5a80-408e-b435-49664bdb23bc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:29.737637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 202/419] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:29.893702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d41006-5a80-408e-b435-49664bdb23bc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:29.903611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:29.929860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584032d-533d-4df7-bb06-ff321a897e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:29.930812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584032d-533d-4df7-bb06-ff321a897e9f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:29.931297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584032d-533d-4df7-bb06-ff321a897e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:29.931746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584032d-533d-4df7-bb06-ff321a897e9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:29.947500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584032d-533d-4df7-bb06-ff321a897e9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-6584032d-533d-4df7-bb06-ff321a897e9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:29.952698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:29.953599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:29.954483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:29.956623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 218/420] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:29.962231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8482db-af17-4a06-96c8-c5c476cbbdb0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:29.964889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:30.058147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584032d-533d-4df7-bb06-ff321a897e9f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:30.115806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8482db-af17-4a06-96c8-c5c476cbbdb0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:30.181300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60597d75-e927-4c58-b3d6-aa98ff64f92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:30.181924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60597d75-e927-4c58-b3d6-aa98ff64f92c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:30.182204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60597d75-e927-4c58-b3d6-aa98ff64f92c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:30.182516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60597d75-e927-4c58-b3d6-aa98ff64f92c None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:30.183221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60597d75-e927-4c58-b3d6-aa98ff64f92c None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:30.183530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60597d75-e927-4c58-b3d6-aa98ff64f92c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:30.184920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60597d75-e927-4c58-b3d6-aa98ff64f92c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:30.201702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 203/421] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:30.207540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60597d75-e927-4c58-b3d6-aa98ff64f92c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:30.227443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584032d-533d-4df7-bb06-ff321a897e9f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:30.298335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca502-f037-4464-a462-cfd5ac2871b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:30.298999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca502-f037-4464-a462-cfd5ac2871b6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:30.299273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca502-f037-4464-a462-cfd5ac2871b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:30.299546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca502-f037-4464-a462-cfd5ac2871b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:30.324720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ca502-f037-4464-a462-cfd5ac2871b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-a42ca502-f037-4464-a462-cfd5ac2871b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:30.331778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:30.332831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:30.333640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 219/422] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:46:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:30.339843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca502-f037-4464-a462-cfd5ac2871b6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:30.352274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:30.375346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:30.456583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca502-f037-4464-a462-cfd5ac2871b6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:30.480710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60597d75-e927-4c58-b3d6-aa98ff64f92c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:30.495377 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91730ef7-a723-423a-af60-746046b10741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:30.496071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91730ef7-a723-423a-af60-746046b10741 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:30.496348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91730ef7-a723-423a-af60-746046b10741 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:30.496616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91730ef7-a723-423a-af60-746046b10741 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:30.497348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91730ef7-a723-423a-af60-746046b10741 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:30.498255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91730ef7-a723-423a-af60-746046b10741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:30.499721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91730ef7-a723-423a-af60-746046b10741 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:30.509448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 220/423] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:30.517224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91730ef7-a723-423a-af60-746046b10741 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:30.562192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e529e8e-6e60-4aef-92cf-e93af9010b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:30.564745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e529e8e-6e60-4aef-92cf-e93af9010b48 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:30.565027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e529e8e-6e60-4aef-92cf-e93af9010b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:30.565296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e529e8e-6e60-4aef-92cf-e93af9010b48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:30.706049 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e529e8e-6e60-4aef-92cf-e93af9010b48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:30.707545 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e529e8e-6e60-4aef-92cf-e93af9010b48 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:30.709669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:30.720799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91730ef7-a723-423a-af60-746046b10741 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:30.771446 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef89ac-20e9-410a-8094-626c4a2c5aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:30.772064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef89ac-20e9-410a-8094-626c4a2c5aef None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:30.772340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef89ac-20e9-410a-8094-626c4a2c5aef None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:30.772604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef89ac-20e9-410a-8094-626c4a2c5aef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:30.835275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:30.874003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef89ac-20e9-410a-8094-626c4a2c5aef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:30.875537 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef89ac-20e9-410a-8094-626c4a2c5aef None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:30.908434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 204/424] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:30.914874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e529e8e-6e60-4aef-92cf-e93af9010b48 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:30.970538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:30.984255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 221/425] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 07:46:30.989436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef89ac-20e9-410a-8094-626c4a2c5aef None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:31.072413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e529e8e-6e60-4aef-92cf-e93af9010b48 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:31.102311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef89ac-20e9-410a-8094-626c4a2c5aef None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:31.108838 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767fad3-f8e0-4da1-8f1f-854fcbf4352c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:31.109477 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767fad3-f8e0-4da1-8f1f-854fcbf4352c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:31.109781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767fad3-f8e0-4da1-8f1f-854fcbf4352c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:31.110126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767fad3-f8e0-4da1-8f1f-854fcbf4352c None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:31.111272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0767fad3-f8e0-4da1-8f1f-854fcbf4352c None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:31.112818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0767fad3-f8e0-4da1-8f1f-854fcbf4352c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:31.114614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 205/426] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:30 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:31.115786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa54b8-ba53-41dd-8e66-6424ba13b141 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:31.116044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa54b8-ba53-41dd-8e66-6424ba13b141 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:31.116406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa54b8-ba53-41dd-8e66-6424ba13b141 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:31.121880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 206/427] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:31.122666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa54b8-ba53-41dd-8e66-6424ba13b141 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:31.138437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf47f4e-8210-4b65-96c7-a0073f8b33d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:31.139205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf47f4e-8210-4b65-96c7-a0073f8b33d1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:31.139492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf47f4e-8210-4b65-96c7-a0073f8b33d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:31.139825 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf47f4e-8210-4b65-96c7-a0073f8b33d1 None admin] PATH_INFO: `/v3/projects/1231532f5f8e4acab98360e05e879050` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:31.140997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf47f4e-8210-4b65-96c7-a0073f8b33d1 None admin] RBAC: Authorizing `identity:get_project(project_id=1231532f5f8e4acab98360e05e879050)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:31.142657 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf47f4e-8210-4b65-96c7-a0073f8b33d1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:31.144585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 222/428] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:30 2022] GET /identity/v3/projects/1231532f5f8e4acab98360e05e879050 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:31.145779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63768c5d-3adb-45d3-af4f-cbca0e8a7c7c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:31.146065 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63768c5d-3adb-45d3-af4f-cbca0e8a7c7c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:31.146342 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63768c5d-3adb-45d3-af4f-cbca0e8a7c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:31.159484 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:31.167977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-63768c5d-3adb-45d3-af4f-cbca0e8a7c7c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:31.169229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-63768c5d-3adb-45d3-af4f-cbca0e8a7c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:31.226009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:31.235651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa54b8-ba53-41dd-8e66-6424ba13b141 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:31.242477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 223/429] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:31.242477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63768c5d-3adb-45d3-af4f-cbca0e8a7c7c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:31.295748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d7fc1-6177-4390-a3c8-c7d127598768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:31.296716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d7fc1-6177-4390-a3c8-c7d127598768 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:31.297192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d7fc1-6177-4390-a3c8-c7d127598768 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:31.297638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d7fc1-6177-4390-a3c8-c7d127598768 None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:31.298947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d7fc1-6177-4390-a3c8-c7d127598768 None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:31.300588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d7fc1-6177-4390-a3c8-c7d127598768 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:31.310701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 207/430] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:31 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:31.311783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d7fc1-6177-4390-a3c8-c7d127598768 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:31.352065 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63768c5d-3adb-45d3-af4f-cbca0e8a7c7c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:31.410274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70aea1-cf7b-4159-9078-1483e6991bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:31.411260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70aea1-cf7b-4159-9078-1483e6991bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:31.413125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70aea1-cf7b-4159-9078-1483e6991bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:31.413610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70aea1-cf7b-4159-9078-1483e6991bd1 None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:31.415757 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70aea1-cf7b-4159-9078-1483e6991bd1 None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:31.419030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70aea1-cf7b-4159-9078-1483e6991bd1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:31.422717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 224/431] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:31 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:31.428699 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70aea1-cf7b-4159-9078-1483e6991bd1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:31.453650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:31.455735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d7fc1-6177-4390-a3c8-c7d127598768 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:31.521795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7192b7a-4ace-41a5-a8ca-994a2b2df400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:31.522703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192b7a-4ace-41a5-a8ca-994a2b2df400 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:31.523908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192b7a-4ace-41a5-a8ca-994a2b2df400 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:31.527056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192b7a-4ace-41a5-a8ca-994a2b2df400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:31.527975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7192b7a-4ace-41a5-a8ca-994a2b2df400 None admin] RBAC: Adding query filter params (group.id=a6936cdfd246457193c0e4a4c5faca91, role.id=991609c9013b45b3907788ef9b42d051, scope.project.id=1231532f5f8e4acab98360e05e879050) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:31.528475 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7192b7a-4ace-41a5-a8ca-994a2b2df400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:31.530084 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7192b7a-4ace-41a5-a8ca-994a2b2df400 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:31.537080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:31.543122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 208/432] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Sep 16 07:46:31 2022] GET /identity/v3/role_assignments?group.id=a6936cdfd246457193c0e4a4c5faca91&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=991609c9013b45b3907788ef9b42d051 => generated 611 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:31.609985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70aea1-cf7b-4159-9078-1483e6991bd1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:31.648241 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d94968-810b-4dda-a36f-6bfb9ce8c1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:31.649166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d94968-810b-4dda-a36f-6bfb9ce8c1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:31.649613 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d94968-810b-4dda-a36f-6bfb9ce8c1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:31.650077 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d94968-810b-4dda-a36f-6bfb9ce8c1b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:31.651538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d94968-810b-4dda-a36f-6bfb9ce8c1b6 None admin] RBAC: Adding query filter params (group.id=a6936cdfd246457193c0e4a4c5faca91, role.id=64e24908c05145b49cc926880ab769cf, scope.project.id=1231532f5f8e4acab98360e05e879050) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:31.652042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d94968-810b-4dda-a36f-6bfb9ce8c1b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:31.653692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d94968-810b-4dda-a36f-6bfb9ce8c1b6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:31.660459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 225/433] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Sep 16 07:46:31 2022] GET /identity/v3/role_assignments?group.id=a6936cdfd246457193c0e4a4c5faca91&scope.project.id=1231532f5f8e4acab98360e05e879050&role.id=64e24908c05145b49cc926880ab769cf => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:31.703212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7192b7a-4ace-41a5-a8ca-994a2b2df400 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:31.715134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b526f-114a-4f56-a4f0-16e23970aabf None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:31.718407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b526f-114a-4f56-a4f0-16e23970aabf None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:31.718956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b526f-114a-4f56-a4f0-16e23970aabf None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:31.723147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 226/434] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:31.728018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded9ba-a210-4c84-80dd-900c22e0e1ca None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:31.728580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded9ba-a210-4c84-80dd-900c22e0e1ca None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:31.729025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded9ba-a210-4c84-80dd-900c22e0e1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:31.757582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f1ded9ba-a210-4c84-80dd-900c22e0e1ca None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:31.758946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f1ded9ba-a210-4c84-80dd-900c22e0e1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:31.833415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7192b7a-4ace-41a5-a8ca-994a2b2df400 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:31.845591 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 227/435] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:31.880579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45aef30-8950-4963-9680-c93865cf6624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:31.881228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45aef30-8950-4963-9680-c93865cf6624 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:31.881505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45aef30-8950-4963-9680-c93865cf6624 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:31.881778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45aef30-8950-4963-9680-c93865cf6624 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:31.893874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45aef30-8950-4963-9680-c93865cf6624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a45aef30-8950-4963-9680-c93865cf6624 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:31.898881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:31.906359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:31.907202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 209/436] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:31.908787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ded9ba-a210-4c84-80dd-900c22e0e1ca None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:31.975303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:32.034880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ded9ba-a210-4c84-80dd-900c22e0e1ca None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:32.049179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fc04c-2466-4cb1-b41a-1d600f60dbbd None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:32.049464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fc04c-2466-4cb1-b41a-1d600f60dbbd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:32.049730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fc04c-2466-4cb1-b41a-1d600f60dbbd None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:32.051777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 210/437] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:32.057385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849a8a-f52f-42be-a916-7060ee9b0d6f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:32.057665 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849a8a-f52f-42be-a916-7060ee9b0d6f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:32.057937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849a8a-f52f-42be-a916-7060ee9b0d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:32.078858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67654c5f-53e0-4565-98ed-9f73e464d6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:32.079853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67654c5f-53e0-4565-98ed-9f73e464d6bc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:32.080134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67654c5f-53e0-4565-98ed-9f73e464d6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:32.080405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67654c5f-53e0-4565-98ed-9f73e464d6bc None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:32.081078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67654c5f-53e0-4565-98ed-9f73e464d6bc None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:32.082136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67654c5f-53e0-4565-98ed-9f73e464d6bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:32.082829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67654c5f-53e0-4565-98ed-9f73e464d6bc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:32.087802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fb849a8a-f52f-42be-a916-7060ee9b0d6f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:32.089601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fb849a8a-f52f-42be-a916-7060ee9b0d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:32.092894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 228/438] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:46:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:32.099676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67654c5f-53e0-4565-98ed-9f73e464d6bc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:32.116664 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:32.172025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 211/439] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:32.173239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb849a8a-f52f-42be-a916-7060ee9b0d6f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:32.223075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67654c5f-53e0-4565-98ed-9f73e464d6bc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:32.241324 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:32.278383 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c90121-a71c-4fe9-b007-23b9e4037ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:32.279380 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c90121-a71c-4fe9-b007-23b9e4037ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:32.279853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c90121-a71c-4fe9-b007-23b9e4037ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:32.280310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c90121-a71c-4fe9-b007-23b9e4037ea1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:32.305619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb849a8a-f52f-42be-a916-7060ee9b0d6f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:32.344130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791be109-ab81-4731-932e-8abd85a835df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:32.344697 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791be109-ab81-4731-932e-8abd85a835df None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:32.344977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791be109-ab81-4731-932e-8abd85a835df None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:32.345697 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791be109-ab81-4731-932e-8abd85a835df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:32.362160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791be109-ab81-4731-932e-8abd85a835df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:32.364638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c90121-a71c-4fe9-b007-23b9e4037ea1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:32.366265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c90121-a71c-4fe9-b007-23b9e4037ea1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-791be109-ab81-4731-932e-8abd85a835df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:32.367560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:32.368527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:32.369339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 212/440] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:46:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:32.376526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791be109-ab81-4731-932e-8abd85a835df None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:32.395139 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:32.488772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 229/441] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:32.489610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c90121-a71c-4fe9-b007-23b9e4037ea1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:32.503363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:32.505938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791be109-ab81-4731-932e-8abd85a835df None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:32.539031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c5fbc-622b-4b65-be4f-4132ed517242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:32.539637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c5fbc-622b-4b65-be4f-4132ed517242 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:32.539913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c5fbc-622b-4b65-be4f-4132ed517242 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:32.540182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c5fbc-622b-4b65-be4f-4132ed517242 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:32.540850 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c5fbc-622b-4b65-be4f-4132ed517242 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:32.541153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c5fbc-622b-4b65-be4f-4132ed517242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:32.542551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c5fbc-622b-4b65-be4f-4132ed517242 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:32.550120 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 213/442] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:32.550919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c5fbc-622b-4b65-be4f-4132ed517242 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:32.592068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c90121-a71c-4fe9-b007-23b9e4037ea1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:32.624243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:32.625954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf6238-d8f6-44bb-908d-5e2fba07fa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:32.626580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf6238-d8f6-44bb-908d-5e2fba07fa39 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:32.626891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf6238-d8f6-44bb-908d-5e2fba07fa39 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:32.627165 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf6238-d8f6-44bb-908d-5e2fba07fa39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:32.637466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf6238-d8f6-44bb-908d-5e2fba07fa39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-dfaf6238-d8f6-44bb-908d-5e2fba07fa39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:32.642274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:32.643357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 230/443] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:46:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:32.644356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf6238-d8f6-44bb-908d-5e2fba07fa39 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:32.649114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c5fbc-622b-4b65-be4f-4132ed517242 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:32.680991 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a46e0c-f80d-4368-b686-beaae548ed9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:32.681613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a46e0c-f80d-4368-b686-beaae548ed9b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:32.681889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a46e0c-f80d-4368-b686-beaae548ed9b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:32.682659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a46e0c-f80d-4368-b686-beaae548ed9b None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:32.683786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a46e0c-f80d-4368-b686-beaae548ed9b None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:32.685235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a46e0c-f80d-4368-b686-beaae548ed9b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:32.686846 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 214/444] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:32 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:32.687595 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a46e0c-f80d-4368-b686-beaae548ed9b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:32.719317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:32.752001 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf6238-d8f6-44bb-908d-5e2fba07fa39 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:32.785776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6735ea8-c508-42b2-9361-753096806bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:32.786379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6735ea8-c508-42b2-9361-753096806bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:32.786657 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6735ea8-c508-42b2-9361-753096806bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:32.786960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6735ea8-c508-42b2-9361-753096806bf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:32.788731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a46e0c-f80d-4368-b686-beaae548ed9b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:32.822637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77881bb-2692-4920-a9a5-2edea7973bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:32.823290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77881bb-2692-4920-a9a5-2edea7973bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:32.823568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77881bb-2692-4920-a9a5-2edea7973bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:32.823838 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77881bb-2692-4920-a9a5-2edea7973bf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:32.824497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77881bb-2692-4920-a9a5-2edea7973bf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:32.824804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77881bb-2692-4920-a9a5-2edea7973bf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:32.826170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77881bb-2692-4920-a9a5-2edea7973bf8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:32.834098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 215/445] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:32.834923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77881bb-2692-4920-a9a5-2edea7973bf8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:32.844170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:32.855678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6735ea8-c508-42b2-9361-753096806bf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:32.857140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6735ea8-c508-42b2-9361-753096806bf4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:32.924722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:32.933496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77881bb-2692-4920-a9a5-2edea7973bf8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:32.955102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 231/446] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:32.958868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6735ea8-c508-42b2-9361-753096806bf4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:32.966732 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db3ed2-e4bd-46cf-a19f-c01e64799775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:32.967372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db3ed2-e4bd-46cf-a19f-c01e64799775 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:32.967658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db3ed2-e4bd-46cf-a19f-c01e64799775 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:32.967932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db3ed2-e4bd-46cf-a19f-c01e64799775 None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:32.969082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db3ed2-e4bd-46cf-a19f-c01e64799775 None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:32.970523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db3ed2-e4bd-46cf-a19f-c01e64799775 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:32.972129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 216/447] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:32 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:32.972886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db3ed2-e4bd-46cf-a19f-c01e64799775 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:33.045966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:33.058329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6735ea8-c508-42b2-9361-753096806bf4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:33.073175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db3ed2-e4bd-46cf-a19f-c01e64799775 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:33.091043 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77819b-c2e8-4ed4-b5ac-88389694773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:33.091645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77819b-c2e8-4ed4-b5ac-88389694773c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:33.091930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77819b-c2e8-4ed4-b5ac-88389694773c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:33.092200 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77819b-c2e8-4ed4-b5ac-88389694773c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:33.102830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11725834-dd27-455b-a280-14e886430b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:33.103455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11725834-dd27-455b-a280-14e886430b67 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:33.103730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11725834-dd27-455b-a280-14e886430b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:33.104495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11725834-dd27-455b-a280-14e886430b67 None admin] PATH_INFO: `/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:33.105602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11725834-dd27-455b-a280-14e886430b67 None admin] RBAC: Authorizing `identity:get_project(project_id=d64c95c677b44381b0ee5c5ab3c1d15d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:33.107144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11725834-dd27-455b-a280-14e886430b67 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:33.108743 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 217/448] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:32 2022] GET /identity/v3/projects/d64c95c677b44381b0ee5c5ab3c1d15d => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:33.109490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11725834-dd27-455b-a280-14e886430b67 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:33.132746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:33.158014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77819b-c2e8-4ed4-b5ac-88389694773c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:33.159535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77819b-c2e8-4ed4-b5ac-88389694773c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:33.208113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11725834-dd27-455b-a280-14e886430b67 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:33.241232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1627e8-610a-464f-a712-151e2b372f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:33.241828 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1627e8-610a-464f-a712-151e2b372f9a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:33.242103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1627e8-610a-464f-a712-151e2b372f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:33.242373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1627e8-610a-464f-a712-151e2b372f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:33.243121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1627e8-610a-464f-a712-151e2b372f9a None admin] RBAC: Adding query filter params (group.id=a6936cdfd246457193c0e4a4c5faca91, role.id=991609c9013b45b3907788ef9b42d051, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:33.243428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1627e8-610a-464f-a712-151e2b372f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:33.244866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1627e8-610a-464f-a712-151e2b372f9a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:33.250528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 218/449] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Sep 16 07:46:33 2022] GET /identity/v3/role_assignments?group.id=a6936cdfd246457193c0e4a4c5faca91&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=991609c9013b45b3907788ef9b42d051 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:33.251143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1627e8-610a-464f-a712-151e2b372f9a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:33.259999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:33.273564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 232/450] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:33.280062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77819b-c2e8-4ed4-b5ac-88389694773c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:33.358719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:33.369501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1627e8-610a-464f-a712-151e2b372f9a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:33.397276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77819b-c2e8-4ed4-b5ac-88389694773c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:33.408000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e685b7a-bafb-4c58-babd-41b37c3fdf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:33.408651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e685b7a-bafb-4c58-babd-41b37c3fdf6d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:33.408929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e685b7a-bafb-4c58-babd-41b37c3fdf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:33.409208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e685b7a-bafb-4c58-babd-41b37c3fdf6d None admin] PATH_INFO: `/v3/groups/a6936cdfd246457193c0e4a4c5faca91` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:33.410378 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e685b7a-bafb-4c58-babd-41b37c3fdf6d None admin] RBAC: Authorizing `identity:get_group(group_id=a6936cdfd246457193c0e4a4c5faca91)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:33.411877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e685b7a-bafb-4c58-babd-41b37c3fdf6d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:33.413671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 219/451] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:33 2022] GET /identity/v3/groups/a6936cdfd246457193c0e4a4c5faca91 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:33.419927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e685b7a-bafb-4c58-babd-41b37c3fdf6d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:33.443637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92041dac-ed6d-4d80-bbfe-413e099dc82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:33.444282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92041dac-ed6d-4d80-bbfe-413e099dc82e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:33.444562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92041dac-ed6d-4d80-bbfe-413e099dc82e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:33.444839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92041dac-ed6d-4d80-bbfe-413e099dc82e None admin] PATH_INFO: `/v3/projects/1dfa255cacf3473b9f6619fec53b8cf1` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:33.445981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92041dac-ed6d-4d80-bbfe-413e099dc82e None admin] RBAC: Authorizing `identity:get_project(project_id=1dfa255cacf3473b9f6619fec53b8cf1)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:33.447497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92041dac-ed6d-4d80-bbfe-413e099dc82e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:33.449181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 233/452] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:46:33 2022] GET /identity/v3/projects/1dfa255cacf3473b9f6619fec53b8cf1 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:33.455280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92041dac-ed6d-4d80-bbfe-413e099dc82e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:33.504594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:33.536694 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e685b7a-bafb-4c58-babd-41b37c3fdf6d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:33.581757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fd4dc-a3c8-4174-a4d1-69b52e42fb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:33.582364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fd4dc-a3c8-4174-a4d1-69b52e42fb1b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:33.582639 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fd4dc-a3c8-4174-a4d1-69b52e42fb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:33.582942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fd4dc-a3c8-4174-a4d1-69b52e42fb1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:33.583573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92041dac-ed6d-4d80-bbfe-413e099dc82e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:33.583712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fd4dc-a3c8-4174-a4d1-69b52e42fb1b None admin] RBAC: Adding query filter params (group.id=a6936cdfd246457193c0e4a4c5faca91, role.id=64e24908c05145b49cc926880ab769cf, scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:33.583960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fd4dc-a3c8-4174-a4d1-69b52e42fb1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:33.585388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fd4dc-a3c8-4174-a4d1-69b52e42fb1b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:33.591554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 220/453] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Sep 16 07:46:33 2022] GET /identity/v3/role_assignments?group.id=a6936cdfd246457193c0e4a4c5faca91&scope.project.id=d64c95c677b44381b0ee5c5ab3c1d15d&role.id=64e24908c05145b49cc926880ab769cf => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:33.624507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a636fb-8b2f-45d4-9466-9adb407b9fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:33.625167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a636fb-8b2f-45d4-9466-9adb407b9fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:33.625443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a636fb-8b2f-45d4-9466-9adb407b9fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:33.625715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a636fb-8b2f-45d4-9466-9adb407b9fc6 None admin] PATH_INFO: `/v3/groups/f4b575263e6e4d779f9cd742523b476e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:33.626960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a636fb-8b2f-45d4-9466-9adb407b9fc6 None admin] RBAC: Authorizing `identity:get_group(group_id=f4b575263e6e4d779f9cd742523b476e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:33.628463 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a636fb-8b2f-45d4-9466-9adb407b9fc6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:33.630117 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 234/454] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Sep 16 07:46:33 2022] GET /identity/v3/groups/f4b575263e6e4d779f9cd742523b476e => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:33.636240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fd4dc-a3c8-4174-a4d1-69b52e42fb1b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:33.745899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fd4dc-a3c8-4174-a4d1-69b52e42fb1b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:33.779818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f77f36-66fe-4c0c-9a1e-cbc7d6b2b55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:33.780350 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f77f36-66fe-4c0c-9a1e-cbc7d6b2b55e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:33.780633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f77f36-66fe-4c0c-9a1e-cbc7d6b2b55e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:33.780897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f77f36-66fe-4c0c-9a1e-cbc7d6b2b55e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:33.781616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f77f36-66fe-4c0c-9a1e-cbc7d6b2b55e None admin] RBAC: Adding query filter params (group.id=f4b575263e6e4d779f9cd742523b476e, role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=1dfa255cacf3473b9f6619fec53b8cf1) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:33.781921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f77f36-66fe-4c0c-9a1e-cbc7d6b2b55e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:33.783444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f77f36-66fe-4c0c-9a1e-cbc7d6b2b55e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:33.789185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 221/455] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Sep 16 07:46:33 2022] GET /identity/v3/role_assignments?group.id=f4b575263e6e4d779f9cd742523b476e&scope.project.id=1dfa255cacf3473b9f6619fec53b8cf1&role.id=62eb3e525077429aacc0ada63300802e => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:36.572471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95351b-7834-49d0-beb5-3e3555d741d5 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:36.573068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95351b-7834-49d0-beb5-3e3555d741d5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:36.573068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95351b-7834-49d0-beb5-3e3555d741d5 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:36.574627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 235/456] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:36.580260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6859e-faf5-446f-87cc-f77d2b04a058 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:36.581078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6859e-faf5-446f-87cc-f77d2b04a058 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:36.581583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6859e-faf5-446f-87cc-f77d2b04a058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:36.594129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:36.608882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-45b6859e-faf5-446f-87cc-f77d2b04a058 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:36.610429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-45b6859e-faf5-446f-87cc-f77d2b04a058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:36.650221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028f3a2-a647-4bc6-9e18-2b5e359599f4 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:36.650577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028f3a2-a647-4bc6-9e18-2b5e359599f4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:36.650826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028f3a2-a647-4bc6-9e18-2b5e359599f4 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:36.652456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 236/457] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:46:36.658396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b5462-ee99-4f46-ae44-8db7966d12dd None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:36.658713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b5462-ee99-4f46-ae44-8db7966d12dd None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:36.658981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b5462-ee99-4f46-ae44-8db7966d12dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:36.689122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-615b5462-ee99-4f46-ae44-8db7966d12dd None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:36.691831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-615b5462-ee99-4f46-ae44-8db7966d12dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:36.694969 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:36.746701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 222/458] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:36.756246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c713ce7-b106-4cf0-a7a2-be5416b260e2 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:36.756528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c713ce7-b106-4cf0-a7a2-be5416b260e2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:36.756803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c713ce7-b106-4cf0-a7a2-be5416b260e2 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:36.758144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 223/459] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:36.763086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf085b-ed4b-436a-99f0-045a19b7f7ee None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:36.763362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf085b-ed4b-436a-99f0-045a19b7f7ee None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:36.763629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf085b-ed4b-436a-99f0-045a19b7f7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:36.781354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-87bf085b-ed4b-436a-99f0-045a19b7f7ee None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:36.782545 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-87bf085b-ed4b-436a-99f0-045a19b7f7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:36.826328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 237/460] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:36.887114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 224/461] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:37.014079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b5462-ee99-4f46-ae44-8db7966d12dd None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:37.100682 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf085b-ed4b-436a-99f0-045a19b7f7ee None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:37.143797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b5462-ee99-4f46-ae44-8db7966d12dd None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:37.200482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:37.206427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2e713-f1cf-4cb2-8241-72d05d8a9320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:37.207104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2e713-f1cf-4cb2-8241-72d05d8a9320 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:37.207387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2e713-f1cf-4cb2-8241-72d05d8a9320 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:37.207663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2e713-f1cf-4cb2-8241-72d05d8a9320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:37.218328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:37.222781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2e713-f1cf-4cb2-8241-72d05d8a9320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:37.224403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2e713-f1cf-4cb2-8241-72d05d8a9320 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-5eb2e713-f1cf-4cb2-8241-72d05d8a9320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:37.241192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:37.242329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:37.243433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:37.243433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:37.243433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:37.244820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 238/462] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:37.250201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20724d4c-c10a-4a3b-b4a7-f517653136b7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:37.250201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20724d4c-c10a-4a3b-b4a7-f517653136b7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:37.250201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20724d4c-c10a-4a3b-b4a7-f517653136b7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:37.250201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 239/463] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:37.253827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20724d4c-c10a-4a3b-b4a7-f517653136b7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:37.254364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf085b-ed4b-436a-99f0-045a19b7f7ee None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:37.293438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361942-840c-4a13-b3ee-bb436b5cd1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:37.294174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361942-840c-4a13-b3ee-bb436b5cd1cb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:37.294573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361942-840c-4a13-b3ee-bb436b5cd1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:37.294967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361942-840c-4a13-b3ee-bb436b5cd1cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:37.306374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05361942-840c-4a13-b3ee-bb436b5cd1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:37.307643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05361942-840c-4a13-b3ee-bb436b5cd1cb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-05361942-840c-4a13-b3ee-bb436b5cd1cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:37.324193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:37.325078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:37.325926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:37.325926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:37.325926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:37.325926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 225/464] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:37.326870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988e1d4-0563-496e-91c7-4c3adb46dc58 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:37.326870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988e1d4-0563-496e-91c7-4c3adb46dc58 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:37.327038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988e1d4-0563-496e-91c7-4c3adb46dc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:37.345627 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3988e1d4-0563-496e-91c7-4c3adb46dc58 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:37.347117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3988e1d4-0563-496e-91c7-4c3adb46dc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:37.377680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20724d4c-c10a-4a3b-b4a7-f517653136b7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:37.414454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7577394-6198-4708-a19b-3251246e686f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:37.415102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7577394-6198-4708-a19b-3251246e686f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:37.415379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7577394-6198-4708-a19b-3251246e686f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:37.415649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7577394-6198-4708-a19b-3251246e686f None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:37.416286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7577394-6198-4708-a19b-3251246e686f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:37.416624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7577394-6198-4708-a19b-3251246e686f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:37.422724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 226/465] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:37.422867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7577394-6198-4708-a19b-3251246e686f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:37.422982 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988e1d4-0563-496e-91c7-4c3adb46dc58 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:37.436860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 240/466] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:37.438368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7577394-6198-4708-a19b-3251246e686f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:37.485664 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:37.499894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:37.557295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7577394-6198-4708-a19b-3251246e686f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:37.557591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988e1d4-0563-496e-91c7-4c3adb46dc58 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:37.596514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37246d-9104-425f-b79a-ceaf7229a93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:37.597273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37246d-9104-425f-b79a-ceaf7229a93e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:37.597895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37246d-9104-425f-b79a-ceaf7229a93e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:37.598721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37246d-9104-425f-b79a-ceaf7229a93e None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:37.599831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37246d-9104-425f-b79a-ceaf7229a93e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:37.600331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37246d-9104-425f-b79a-ceaf7229a93e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:37.601907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37246d-9104-425f-b79a-ceaf7229a93e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:37.603095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08e32a-1bc7-496d-8a07-6e2acbdbf6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:37.603789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e32a-1bc7-496d-8a07-6e2acbdbf6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:37.604125 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e32a-1bc7-496d-8a07-6e2acbdbf6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:37.604692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e32a-1bc7-496d-8a07-6e2acbdbf6b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:37.618187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 241/467] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:37.618351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08e32a-1bc7-496d-8a07-6e2acbdbf6b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:37.619519 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37246d-9104-425f-b79a-ceaf7229a93e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:37.620138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08e32a-1bc7-496d-8a07-6e2acbdbf6b5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-bf08e32a-1bc7-496d-8a07-6e2acbdbf6b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:37.636164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:37.637067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:37.638027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:37.638027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:37.638027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:37.640213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 227/468] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:37.641203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08e32a-1bc7-496d-8a07-6e2acbdbf6b5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:37.736233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37246d-9104-425f-b79a-ceaf7229a93e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:37.740218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:37.746718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08e32a-1bc7-496d-8a07-6e2acbdbf6b5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:37.751143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:37.775765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19870dda-97f6-4a09-b2dd-9a42b5fcab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:37.776367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19870dda-97f6-4a09-b2dd-9a42b5fcab8f None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:37.776643 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19870dda-97f6-4a09-b2dd-9a42b5fcab8f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:37.776998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19870dda-97f6-4a09-b2dd-9a42b5fcab8f None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:37.777633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19870dda-97f6-4a09-b2dd-9a42b5fcab8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:37.779094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19870dda-97f6-4a09-b2dd-9a42b5fcab8f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:37.783480 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776b697-487a-4fcf-9812-1c0607030c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:37.784087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776b697-487a-4fcf-9812-1c0607030c91 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:37.784367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776b697-487a-4fcf-9812-1c0607030c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:37.784639 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776b697-487a-4fcf-9812-1c0607030c91 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:37.785391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776b697-487a-4fcf-9812-1c0607030c91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:37.786857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776b697-487a-4fcf-9812-1c0607030c91 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:37.874814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 242/469] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:46:37 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:46:37.875660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19870dda-97f6-4a09-b2dd-9a42b5fcab8f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:37.913160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 228/470] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:46:37 2022] POST /identity/v3/users => generated 246 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:46:37.914423 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776b697-487a-4fcf-9812-1c0607030c91 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:38.015830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19870dda-97f6-4a09-b2dd-9a42b5fcab8f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:38.069930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693439c-0725-4296-8f74-1ac3407f1ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:38.071231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693439c-0725-4296-8f74-1ac3407f1ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:38.071752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693439c-0725-4296-8f74-1ac3407f1ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:38.072400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693439c-0725-4296-8f74-1ac3407f1ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:38.073379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693439c-0725-4296-8f74-1ac3407f1ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:38.073692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693439c-0725-4296-8f74-1ac3407f1ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:38.075149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693439c-0725-4296-8f74-1ac3407f1ae9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:38.090817 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776b697-487a-4fcf-9812-1c0607030c91 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:38.094805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 243/471] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:38.096338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919337-e719-45c8-867f-145a2b3dd410 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:38.096624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919337-e719-45c8-867f-145a2b3dd410 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:38.096901 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919337-e719-45c8-867f-145a2b3dd410 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:38.098467 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 244/472] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:38.099138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c919337-e719-45c8-867f-145a2b3dd410 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:38.132469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:38.144517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a1f64-f7ea-4251-b32f-8821aaf38c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:38.145169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a1f64-f7ea-4251-b32f-8821aaf38c60 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:38.149560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a1f64-f7ea-4251-b32f-8821aaf38c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:38.149560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a1f64-f7ea-4251-b32f-8821aaf38c60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:38.157823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a1f64-f7ea-4251-b32f-8821aaf38c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:38.159331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a1f64-f7ea-4251-b32f-8821aaf38c60 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ea0a1f64-f7ea-4251-b32f-8821aaf38c60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:38.176608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:38.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:38.184544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:38.184544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:38.184544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:38.184544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 229/473] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:38.184544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8b8be-d735-41a1-b0dd-6dfad0efad5a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:38.184544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8b8be-d735-41a1-b0dd-6dfad0efad5a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:38.184544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8b8be-d735-41a1-b0dd-6dfad0efad5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:38.190719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:38.202087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-83a8b8be-d735-41a1-b0dd-6dfad0efad5a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:38.203317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-83a8b8be-d735-41a1-b0dd-6dfad0efad5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:38.223301 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c919337-e719-45c8-867f-145a2b3dd410 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:38.263251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbc943-580c-4dab-861b-4ad5d729b692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:38.263858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbc943-580c-4dab-861b-4ad5d729b692 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:38.264137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbc943-580c-4dab-861b-4ad5d729b692 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:38.264407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbc943-580c-4dab-861b-4ad5d729b692 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:38.265054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbc943-580c-4dab-861b-4ad5d729b692 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:38.266487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbc943-580c-4dab-861b-4ad5d729b692 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:38.282775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 230/474] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:38.283761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8b8be-d735-41a1-b0dd-6dfad0efad5a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:38.389907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8b8be-d735-41a1-b0dd-6dfad0efad5a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:38.394057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 245/475] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:46:38 2022] POST /identity/v3/users => generated 248 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:46:38.434729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081a544-0099-4e62-aad6-623fe01952ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:38.435270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081a544-0099-4e62-aad6-623fe01952ac None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:38.435548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081a544-0099-4e62-aad6-623fe01952ac None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:38.435818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081a544-0099-4e62-aad6-623fe01952ac None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:38.436471 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081a544-0099-4e62-aad6-623fe01952ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:38.436772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081a544-0099-4e62-aad6-623fe01952ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:38.438404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081a544-0099-4e62-aad6-623fe01952ac None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:38.440133 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:38.453846 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 231/476] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:38.459511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbc943-580c-4dab-861b-4ad5d729b692 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:38.585660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:38.611840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbc943-580c-4dab-861b-4ad5d729b692 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:38.613808 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081a544-0099-4e62-aad6-623fe01952ac None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:38.685512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535819b-4dae-4966-8f26-252c4963565d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:38.686208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535819b-4dae-4966-8f26-252c4963565d None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:38.686645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535819b-4dae-4966-8f26-252c4963565d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:38.687715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535819b-4dae-4966-8f26-252c4963565d None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:38.689125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535819b-4dae-4966-8f26-252c4963565d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:38.690588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535819b-4dae-4966-8f26-252c4963565d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:38.763155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081a544-0099-4e62-aad6-623fe01952ac None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:38.799686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133812c-3468-499b-90d2-5812d7d18b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:38.800354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133812c-3468-499b-90d2-5812d7d18b57 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:38.800633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133812c-3468-499b-90d2-5812d7d18b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:38.800902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133812c-3468-499b-90d2-5812d7d18b57 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:38.801636 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133812c-3468-499b-90d2-5812d7d18b57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:38.803060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133812c-3468-499b-90d2-5812d7d18b57 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:38.809372 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 246/477] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:46:38 2022] POST /identity/v3/users => generated 249 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:46:38.885734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.sql.core [None req-9133812c-3468-499b-90d2-5812d7d18b57 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 16 07:46:38.885734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 16 07:46:38.885734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [parameters: {'id': 'b40dd0baed7a4a3e818e7fa54b5dbaef', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 16 07:46:38.885734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81574) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9133812c-3468-499b-90d2-5812d7d18b57 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application conn.query(q) Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application result.read() Sep 16 07:46:38.902528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return ref.to_dict() Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 07:46:38.903485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application next(self.gen) Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application yield resource Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application next(self.gen) Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self.session.rollback() Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self.force_reraise() Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise self.value Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application session.commit() Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 16 07:46:38.904922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._prepare_impl() Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self.session.flush() Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._flush(objects) Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application compat.raise_( Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application flush_context.execute() Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rec.execute(self) Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 16 07:46:38.905802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 07:46:38.906793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application conn.query(q) Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application result.read() Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 07:46:38.907614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [parameters: {'id': 'b40dd0baed7a4a3e818e7fa54b5dbaef', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:38.908468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 16 07:46:38.909332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:38.910203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 232/478] 10.222.0.59 () {60 vars in 1131 bytes} [Fri Sep 16 07:46:38 2022] POST /identity/v3/roles => generated 144 bytes in 291 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:38.917705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535819b-4dae-4966-8f26-252c4963565d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:39.061698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535819b-4dae-4966-8f26-252c4963565d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:39.098588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef534ac-16df-4f42-a80c-c7a210900055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:39.099618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef534ac-16df-4f42-a80c-c7a210900055 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:39.100107 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef534ac-16df-4f42-a80c-c7a210900055 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:39.100594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef534ac-16df-4f42-a80c-c7a210900055 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:39.107860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:39.113624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef534ac-16df-4f42-a80c-c7a210900055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-2ef534ac-16df-4f42-a80c-c7a210900055 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:39.118805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:39.121416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 07:46:39.122226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:39.123748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 247/479] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:46:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:39.126296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133812c-3468-499b-90d2-5812d7d18b57 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:39.201207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:39.248579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133812c-3468-499b-90d2-5812d7d18b57 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:39.288879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094212a-0b2a-40f3-8fe0-39050735a732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:39.289501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094212a-0b2a-40f3-8fe0-39050735a732 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:39.289784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094212a-0b2a-40f3-8fe0-39050735a732 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:39.290108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094212a-0b2a-40f3-8fe0-39050735a732 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:39.290806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094212a-0b2a-40f3-8fe0-39050735a732 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:39.291112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094212a-0b2a-40f3-8fe0-39050735a732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:39.292526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094212a-0b2a-40f3-8fe0-39050735a732 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:39.306829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 233/480] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:40.388819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb12c8-a89b-4ef5-acb5-0aea6ffa6de7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:40.389359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb12c8-a89b-4ef5-acb5-0aea6ffa6de7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:40.389359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb12c8-a89b-4ef5-acb5-0aea6ffa6de7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:40.390971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 248/481] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:40.396786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba088dc1-3b97-420c-91f5-877b50f95953 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:40.397282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba088dc1-3b97-420c-91f5-877b50f95953 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:40.397385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba088dc1-3b97-420c-91f5-877b50f95953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:40.421764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ba088dc1-3b97-420c-91f5-877b50f95953 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:40.422981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ba088dc1-3b97-420c-91f5-877b50f95953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:40.492904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 234/482] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:40.561107 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bb829-4837-4474-a333-52fe59b4f9f6 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:40.561783 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bb829-4837-4474-a333-52fe59b4f9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:40.562286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bb829-4837-4474-a333-52fe59b4f9f6 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:40.564388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 249/483] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:40.569426 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644469a0-4173-4853-b594-4e70cb064029 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:40.569711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644469a0-4173-4853-b594-4e70cb064029 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:40.569984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644469a0-4173-4853-b594-4e70cb064029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:40.595680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-644469a0-4173-4853-b594-4e70cb064029 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:40.596870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-644469a0-4173-4853-b594-4e70cb064029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:40.653236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:40.679138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 235/484] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:40.815838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bb829-4837-4474-a333-52fe59b4f9f6 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:40.954279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bb829-4837-4474-a333-52fe59b4f9f6 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:41.029109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46493a-a968-4ddf-8833-855481a4d42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:41.029747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46493a-a968-4ddf-8833-855481a4d42a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.030026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46493a-a968-4ddf-8833-855481a4d42a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.030302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46493a-a968-4ddf-8833-855481a4d42a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.047227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46493a-a968-4ddf-8833-855481a4d42a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-aa46493a-a968-4ddf-8833-855481a4d42a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:41.052602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:41.053487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:41.054276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 250/485] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:41.061262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644469a0-4173-4853-b594-4e70cb064029 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:41.128507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46493a-a968-4ddf-8833-855481a4d42a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:41.215131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:41.242323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644469a0-4173-4853-b594-4e70cb064029 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:41.291889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46493a-a968-4ddf-8833-855481a4d42a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:41.308592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1f9eb-7c9b-4969-8440-f19cb2ab6218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:41.309654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1f9eb-7c9b-4969-8440-f19cb2ab6218 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.310468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1f9eb-7c9b-4969-8440-f19cb2ab6218 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.311037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1f9eb-7c9b-4969-8440-f19cb2ab6218 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.311966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1f9eb-7c9b-4969-8440-f19cb2ab6218 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:41.312514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1f9eb-7c9b-4969-8440-f19cb2ab6218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:41.315159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1f9eb-7c9b-4969-8440-f19cb2ab6218 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:41.327573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 236/486] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:41.329115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23929a-32c0-4396-93fb-9bd9627a3ef8 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.329613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23929a-32c0-4396-93fb-9bd9627a3ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.330101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23929a-32c0-4396-93fb-9bd9627a3ef8 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.332045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 237/487] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:41.332352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b320f5-2050-4dcd-80a4-54313181d899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:41.332997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b320f5-2050-4dcd-80a4-54313181d899 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.333280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b320f5-2050-4dcd-80a4-54313181d899 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.333557 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b320f5-2050-4dcd-80a4-54313181d899 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.333831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23929a-32c0-4396-93fb-9bd9627a3ef8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:41.349456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b320f5-2050-4dcd-80a4-54313181d899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-c2b320f5-2050-4dcd-80a4-54313181d899 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:41.354655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:41.355655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:41.356462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:41.360678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 251/488] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:41.361990 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661f3f8-972a-4b90-b8cc-50d6ffbc2b3b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.362335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661f3f8-972a-4b90-b8cc-50d6ffbc2b3b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.362611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661f3f8-972a-4b90-b8cc-50d6ffbc2b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.381131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2661f3f8-972a-4b90-b8cc-50d6ffbc2b3b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:41.381800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2661f3f8-972a-4b90-b8cc-50d6ffbc2b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:41.391464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:41.462929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 252/489] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:41.463884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661f3f8-972a-4b90-b8cc-50d6ffbc2b3b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:41.465035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23929a-32c0-4396-93fb-9bd9627a3ef8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:41.504524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614c929-15a5-4a32-be0a-63f4d358f4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:41.505490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614c929-15a5-4a32-be0a-63f4d358f4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.506108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614c929-15a5-4a32-be0a-63f4d358f4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.506953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614c929-15a5-4a32-be0a-63f4d358f4e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.519753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614c929-15a5-4a32-be0a-63f4d358f4e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:41.521501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614c929-15a5-4a32-be0a-63f4d358f4e3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5614c929-15a5-4a32-be0a-63f4d358f4e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:41.540888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:41.541904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:41.542791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:41.542791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:41.542791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:41.545255 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 238/490] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:41.558196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614c929-15a5-4a32-be0a-63f4d358f4e3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:41.560896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:41.618999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661f3f8-972a-4b90-b8cc-50d6ffbc2b3b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:41.667610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951675e-eed1-49a2-82d4-bf21b13b54df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:41.668224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951675e-eed1-49a2-82d4-bf21b13b54df None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.669889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951675e-eed1-49a2-82d4-bf21b13b54df None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.670177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951675e-eed1-49a2-82d4-bf21b13b54df None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.670923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951675e-eed1-49a2-82d4-bf21b13b54df None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:41.671232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951675e-eed1-49a2-82d4-bf21b13b54df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:41.672596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951675e-eed1-49a2-82d4-bf21b13b54df None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:41.685961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 253/491] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:41.691977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951675e-eed1-49a2-82d4-bf21b13b54df None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:41.732392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:41.739112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614c929-15a5-4a32-be0a-63f4d358f4e3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:41.775309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b6401-d5ed-423d-ab74-06ada848b8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:41.776014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b6401-d5ed-423d-ab74-06ada848b8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.776393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b6401-d5ed-423d-ab74-06ada848b8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.776668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b6401-d5ed-423d-ab74-06ada848b8c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.777306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b6401-d5ed-423d-ab74-06ada848b8c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:41.777606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b6401-d5ed-423d-ab74-06ada848b8c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:41.779107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b6401-d5ed-423d-ab74-06ada848b8c3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:41.791865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 239/492] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:41.793086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038a293-1b2f-476b-9399-25d113e23eb1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.793373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038a293-1b2f-476b-9399-25d113e23eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.793646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038a293-1b2f-476b-9399-25d113e23eb1 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.795031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 240/493] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:41.799637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038a293-1b2f-476b-9399-25d113e23eb1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:41.818250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951675e-eed1-49a2-82d4-bf21b13b54df None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:41.829825 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:41.855695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe238cef-6ce6-4879-883c-4f1c3b2e06a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:41.856773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238cef-6ce6-4879-883c-4f1c3b2e06a0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.857203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238cef-6ce6-4879-883c-4f1c3b2e06a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.857656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238cef-6ce6-4879-883c-4f1c3b2e06a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.870488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe238cef-6ce6-4879-883c-4f1c3b2e06a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:41.871978 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe238cef-6ce6-4879-883c-4f1c3b2e06a0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-fe238cef-6ce6-4879-883c-4f1c3b2e06a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:41.886565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:41.887554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:41.888456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:41.888456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:41.888456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:41.892535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 254/494] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:41.893623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8576d-3987-454a-8bf1-bbd3edffdd74 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.893909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8576d-3987-454a-8bf1-bbd3edffdd74 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.894183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8576d-3987-454a-8bf1-bbd3edffdd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.906728 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038a293-1b2f-476b-9399-25d113e23eb1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:41.913387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-43d8576d-3987-454a-8bf1-bbd3edffdd74 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:41.914595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-43d8576d-3987-454a-8bf1-bbd3edffdd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:41.944335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f0fca-1025-4644-910d-f0175b595828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:41.944980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0fca-1025-4644-910d-f0175b595828 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:41.945258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0fca-1025-4644-910d-f0175b595828 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:41.945533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0fca-1025-4644-910d-f0175b595828 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:41.952820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f0fca-1025-4644-910d-f0175b595828 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:41.954253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f0fca-1025-4644-910d-f0175b595828 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-229f0fca-1025-4644-910d-f0175b595828 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:41.967745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:41.968755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:41.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:41.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:41.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:41.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:41.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:41.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:41.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:41.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:41.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 16 07:46:41.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:41.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 241/495] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:41.970223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f0fca-1025-4644-910d-f0175b595828 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:41.988138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:42.006707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 255/496] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:42.006707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8576d-3987-454a-8bf1-bbd3edffdd74 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:42.080234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f0fca-1025-4644-910d-f0175b595828 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:42.114476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06b8a5-0863-4d74-9911-96d6303e81d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:42.115241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06b8a5-0863-4d74-9911-96d6303e81d8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.115517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06b8a5-0863-4d74-9911-96d6303e81d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.115786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06b8a5-0863-4d74-9911-96d6303e81d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.121037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:42.126985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06b8a5-0863-4d74-9911-96d6303e81d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:42.130037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8576d-3987-454a-8bf1-bbd3edffdd74 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-de06b8a5-0863-4d74-9911-96d6303e81d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:42.132233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:42.133492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 242/497] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:42.134295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06b8a5-0863-4d74-9911-96d6303e81d8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:42.171702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9781fd-8c9e-4103-be46-11eebe186294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:42.172323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9781fd-8c9e-4103-be46-11eebe186294 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.172587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9781fd-8c9e-4103-be46-11eebe186294 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.172861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9781fd-8c9e-4103-be46-11eebe186294 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.173495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9781fd-8c9e-4103-be46-11eebe186294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:42.173800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9781fd-8c9e-4103-be46-11eebe186294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:42.175197 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9781fd-8c9e-4103-be46-11eebe186294 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:42.187071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 256/498] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:42.188681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74fac-e51b-418c-853c-b87175ed0661 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.188945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74fac-e51b-418c-853c-b87175ed0661 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.189218 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74fac-e51b-418c-853c-b87175ed0661 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.190554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 257/499] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:42.191174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b74fac-e51b-418c-853c-b87175ed0661 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:42.218914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:42.242444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06b8a5-0863-4d74-9911-96d6303e81d8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:42.277922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02928a-19b8-430b-8dd3-8dc81b32fb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:42.278527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02928a-19b8-430b-8dd3-8dc81b32fb31 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.278839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02928a-19b8-430b-8dd3-8dc81b32fb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.279113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02928a-19b8-430b-8dd3-8dc81b32fb31 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.279837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02928a-19b8-430b-8dd3-8dc81b32fb31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:42.280142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02928a-19b8-430b-8dd3-8dc81b32fb31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:42.281584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02928a-19b8-430b-8dd3-8dc81b32fb31 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:42.298665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b74fac-e51b-418c-853c-b87175ed0661 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:42.317486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 243/500] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Sep 16 07:46:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:42.318291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02928a-19b8-430b-8dd3-8dc81b32fb31 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:42.336566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e2e68-00a4-4b77-bdd3-d7beea95320f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:42.337114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e2e68-00a4-4b77-bdd3-d7beea95320f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.337394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e2e68-00a4-4b77-bdd3-d7beea95320f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.337666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e2e68-00a4-4b77-bdd3-d7beea95320f None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.338346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e2e68-00a4-4b77-bdd3-d7beea95320f None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:42.338649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e2e68-00a4-4b77-bdd3-d7beea95320f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:42.340033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e2e68-00a4-4b77-bdd3-d7beea95320f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:42.347735 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 258/501] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:42.348857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030365fd-3751-48e2-8438-3d31282ee3e8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.349143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030365fd-3751-48e2-8438-3d31282ee3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.349420 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030365fd-3751-48e2-8438-3d31282ee3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.366364 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-030365fd-3751-48e2-8438-3d31282ee3e8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:42.367590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-030365fd-3751-48e2-8438-3d31282ee3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:42.377076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:42.423032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02928a-19b8-430b-8dd3-8dc81b32fb31 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:42.434761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 259/502] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:42.435585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030365fd-3751-48e2-8438-3d31282ee3e8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:42.464124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:42.470206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c4a04-c605-431e-abff-226f1025c2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:42.471077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c4a04-c605-431e-abff-226f1025c2fe None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.471187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c4a04-c605-431e-abff-226f1025c2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.471404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c4a04-c605-431e-abff-226f1025c2fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.478632 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c4a04-c605-431e-abff-226f1025c2fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:42.480176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c4a04-c605-431e-abff-226f1025c2fe None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-862c4a04-c605-431e-abff-226f1025c2fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:42.491984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:42.493075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:42.494188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:42.494188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:42.494188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:42.494188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:42.494188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:42.494188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:42.494188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:42.494188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:42.494188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:46:42.494188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:42.496637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 244/503] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Sep 16 07:46:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:42.497568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c4a04-c605-431e-abff-226f1025c2fe None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:42.548007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030365fd-3751-48e2-8438-3d31282ee3e8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:42.582759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6a33b-a57a-495b-b7b0-ff0c09af74fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:42.583461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6a33b-a57a-495b-b7b0-ff0c09af74fe None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.583815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6a33b-a57a-495b-b7b0-ff0c09af74fe None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.584149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6a33b-a57a-495b-b7b0-ff0c09af74fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.594317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6a33b-a57a-495b-b7b0-ff0c09af74fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-70b6a33b-a57a-495b-b7b0-ff0c09af74fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:42.599145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:42.600027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:42.600893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 260/504] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:42.601573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6a33b-a57a-495b-b7b0-ff0c09af74fe None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:42.606599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:42.613113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c4a04-c605-431e-abff-226f1025c2fe None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:42.646788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864fa5b-b681-4ed2-918e-741201e54e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:42.647386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864fa5b-b681-4ed2-918e-741201e54e88 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.647658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864fa5b-b681-4ed2-918e-741201e54e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.647930 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864fa5b-b681-4ed2-918e-741201e54e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.707051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6a33b-a57a-495b-b7b0-ff0c09af74fe None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:42.710576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:42.719888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864fa5b-b681-4ed2-918e-741201e54e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:42.725065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864fa5b-b681-4ed2-918e-741201e54e88 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:42.742629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afec9bf-6e73-447c-8744-93865b9139fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:42.743339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afec9bf-6e73-447c-8744-93865b9139fd None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.743618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afec9bf-6e73-447c-8744-93865b9139fd None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.743913 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afec9bf-6e73-447c-8744-93865b9139fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.754621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afec9bf-6e73-447c-8744-93865b9139fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:42.756111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afec9bf-6e73-447c-8744-93865b9139fd None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-4afec9bf-6e73-447c-8744-93865b9139fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:42.771181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:42.772259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:42.773446 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:42.773446 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:42.773446 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:42.773446 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 261/505] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:42.773446 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afec9bf-6e73-447c-8744-93865b9139fd None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:42.824105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 245/506] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:42.825062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864fa5b-b681-4ed2-918e-741201e54e88 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:42.829246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:42.873355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afec9bf-6e73-447c-8744-93865b9139fd None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:42.906979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75800cba-f0b2-4cff-bfb6-9c65fc28f8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:42.907578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75800cba-f0b2-4cff-bfb6-9c65fc28f8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.907853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75800cba-f0b2-4cff-bfb6-9c65fc28f8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.908122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75800cba-f0b2-4cff-bfb6-9c65fc28f8e9 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.908835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75800cba-f0b2-4cff-bfb6-9c65fc28f8e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:42.909137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75800cba-f0b2-4cff-bfb6-9c65fc28f8e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:42.910553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75800cba-f0b2-4cff-bfb6-9c65fc28f8e9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:42.917426 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:42.925854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864fa5b-b681-4ed2-918e-741201e54e88 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:42.947096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 262/507] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Sep 16 07:46:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:42.947916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75800cba-f0b2-4cff-bfb6-9c65fc28f8e9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:42.960546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83373566-bdaa-4110-8aed-669ed919edf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:42.961156 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373566-bdaa-4110-8aed-669ed919edf5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:42.961435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373566-bdaa-4110-8aed-669ed919edf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:42.961710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373566-bdaa-4110-8aed-669ed919edf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:42.962377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83373566-bdaa-4110-8aed-669ed919edf5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:42.962769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83373566-bdaa-4110-8aed-669ed919edf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:42.964585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83373566-bdaa-4110-8aed-669ed919edf5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:42.972290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 246/508] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:42.973070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83373566-bdaa-4110-8aed-669ed919edf5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.044219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75800cba-f0b2-4cff-bfb6-9c65fc28f8e9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:43.057769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:43.073755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83373566-bdaa-4110-8aed-669ed919edf5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:43.077114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fd0b0-0492-430b-9f1f-5bd568b8b90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:43.077744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd0b0-0492-430b-9f1f-5bd568b8b90d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:43.078022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd0b0-0492-430b-9f1f-5bd568b8b90d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:43.078299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd0b0-0492-430b-9f1f-5bd568b8b90d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:43.088786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fd0b0-0492-430b-9f1f-5bd568b8b90d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:43.090331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fd0b0-0492-430b-9f1f-5bd568b8b90d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-a03fd0b0-0492-430b-9f1f-5bd568b8b90d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:43.105610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:43.106528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:43.107441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:43.107441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:43.107441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:43.107441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 263/509] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:43.107660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fd0b0-0492-430b-9f1f-5bd568b8b90d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.112181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fabd8-d2fa-499d-928f-f1aaa88b8977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:43.112796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fabd8-d2fa-499d-928f-f1aaa88b8977 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:43.113076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fabd8-d2fa-499d-928f-f1aaa88b8977 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:43.113349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fabd8-d2fa-499d-928f-f1aaa88b8977 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:43.113990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fabd8-d2fa-499d-928f-f1aaa88b8977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:43.114293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fabd8-d2fa-499d-928f-f1aaa88b8977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:43.115713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fabd8-d2fa-499d-928f-f1aaa88b8977 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:43.129169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 247/510] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:43.129890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fabd8-d2fa-499d-928f-f1aaa88b8977 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.147204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:43.204495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fd0b0-0492-430b-9f1f-5bd568b8b90d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:43.228684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fabd8-d2fa-499d-928f-f1aaa88b8977 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:43.237692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27c884-3c37-4976-a007-d2dbf50fcb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:43.238390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27c884-3c37-4976-a007-d2dbf50fcb8b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:43.238769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27c884-3c37-4976-a007-d2dbf50fcb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:43.239112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27c884-3c37-4976-a007-d2dbf50fcb8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:43.249461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27c884-3c37-4976-a007-d2dbf50fcb8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:43.250949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27c884-3c37-4976-a007-d2dbf50fcb8b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:43.262940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9285ee-901d-4b25-ad2b-8863b0aa09fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:43.263549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9285ee-901d-4b25-ad2b-8863b0aa09fd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:43.263829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9285ee-901d-4b25-ad2b-8863b0aa09fd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:43.264103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9285ee-901d-4b25-ad2b-8863b0aa09fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-5c27c884-3c37-4976-a007-d2dbf50fcb8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:43.264930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:43.265968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:43.266859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:43.266859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:43.266859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:43.266859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 264/511] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:43.266859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27c884-3c37-4976-a007-d2dbf50fcb8b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.287047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:43.335248 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9285ee-901d-4b25-ad2b-8863b0aa09fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:43.336706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9285ee-901d-4b25-ad2b-8863b0aa09fd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:43.351116 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:43.369013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27c884-3c37-4976-a007-d2dbf50fcb8b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:43.402307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea5894-1112-4692-b216-2f0338f3f8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:43.402890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea5894-1112-4692-b216-2f0338f3f8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:43.403177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea5894-1112-4692-b216-2f0338f3f8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:43.403447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea5894-1112-4692-b216-2f0338f3f8b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:43.413577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea5894-1112-4692-b216-2f0338f3f8b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:43.415094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea5894-1112-4692-b216-2f0338f3f8b7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-f6ea5894-1112-4692-b216-2f0338f3f8b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:43.428830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:43.429867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:43.431049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:43.431049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:43.431049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:43.433539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 248/512] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:43.434526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9285ee-901d-4b25-ad2b-8863b0aa09fd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.434644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 265/513] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:43.435335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea5894-1112-4692-b216-2f0338f3f8b7 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.502598 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:43.528648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9285ee-901d-4b25-ad2b-8863b0aa09fd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:43.532583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea5894-1112-4692-b216-2f0338f3f8b7 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:43.543628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:43.563447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb814cd-9cb5-4737-ae4d-f2f467375b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:43.564046 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb814cd-9cb5-4737-ae4d-f2f467375b36 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:43.564365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb814cd-9cb5-4737-ae4d-f2f467375b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:43.564895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb814cd-9cb5-4737-ae4d-f2f467375b36 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:43.565753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb814cd-9cb5-4737-ae4d-f2f467375b36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:43.566312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb814cd-9cb5-4737-ae4d-f2f467375b36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:43.567064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0218ac-e1ac-4a7f-acf8-cc37c3b67f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:43.567696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0218ac-e1ac-4a7f-acf8-cc37c3b67f6b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:43.567803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb814cd-9cb5-4737-ae4d-f2f467375b36 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:43.568005 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0218ac-e1ac-4a7f-acf8-cc37c3b67f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:43.568281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0218ac-e1ac-4a7f-acf8-cc37c3b67f6b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:43.576338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0218ac-e1ac-4a7f-acf8-cc37c3b67f6b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:43.577766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0218ac-e1ac-4a7f-acf8-cc37c3b67f6b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:43.580855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 249/514] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:43.581668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb814cd-9cb5-4737-ae4d-f2f467375b36 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-bf0218ac-e1ac-4a7f-acf8-cc37c3b67f6b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:43.591865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:43.592931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:43.593995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:43.593995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:43.593995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:43.593995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:43.593995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:43.593995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:43.593995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:43.593995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:43.593995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 07:46:43.593995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:43.593995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 266/515] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:43.595738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0218ac-e1ac-4a7f-acf8-cc37c3b67f6b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.680977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb814cd-9cb5-4737-ae4d-f2f467375b36 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:43.692969 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0218ac-e1ac-4a7f-acf8-cc37c3b67f6b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:43.714314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d41af0-03a9-4091-8afd-d10859f0fd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:43.714945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41af0-03a9-4091-8afd-d10859f0fd12 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:43.715223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41af0-03a9-4091-8afd-d10859f0fd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:43.715494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41af0-03a9-4091-8afd-d10859f0fd12 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:43.716123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d41af0-03a9-4091-8afd-d10859f0fd12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:43.716424 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d41af0-03a9-4091-8afd-d10859f0fd12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:43.717807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d41af0-03a9-4091-8afd-d10859f0fd12 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:43.721576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:43.727114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13dafb-d175-44f6-a3d1-8f13f07810c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:43.733456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13dafb-d175-44f6-a3d1-8f13f07810c1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:43.733456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13dafb-d175-44f6-a3d1-8f13f07810c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:43.733456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13dafb-d175-44f6-a3d1-8f13f07810c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:43.733456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13dafb-d175-44f6-a3d1-8f13f07810c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:43.733456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13dafb-d175-44f6-a3d1-8f13f07810c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:43.733456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13dafb-d175-44f6-a3d1-8f13f07810c1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:43.733908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 250/516] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:43.733908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d41af0-03a9-4091-8afd-d10859f0fd12 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.742670 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 267/517] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:43.743499 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13dafb-d175-44f6-a3d1-8f13f07810c1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.772317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:43.836404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d41af0-03a9-4091-8afd-d10859f0fd12 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:43.841225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13dafb-d175-44f6-a3d1-8f13f07810c1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:43.870754 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576300-c3fb-4eb6-9875-64fe2d94419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:43.871381 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576300-c3fb-4eb6-9875-64fe2d94419b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:43.871658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576300-c3fb-4eb6-9875-64fe2d94419b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:43.871932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576300-c3fb-4eb6-9875-64fe2d94419b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:43.873601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0067f22-6723-4ec6-af85-7b826eae81aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:43.874199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0067f22-6723-4ec6-af85-7b826eae81aa None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:43.874475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0067f22-6723-4ec6-af85-7b826eae81aa None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:43.874779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0067f22-6723-4ec6-af85-7b826eae81aa None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:43.875411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0067f22-6723-4ec6-af85-7b826eae81aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:43.876840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0067f22-6723-4ec6-af85-7b826eae81aa None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:43.881827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576300-c3fb-4eb6-9875-64fe2d94419b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:43.883208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576300-c3fb-4eb6-9875-64fe2d94419b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ba576300-c3fb-4eb6-9875-64fe2d94419b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:43.898208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:43.899311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:43.900365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:43.900365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:43.900365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:43.900365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 251/518] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:43.900365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576300-c3fb-4eb6-9875-64fe2d94419b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.954464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 268/519] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:46:43 2022] POST /identity/v3/users => generated 247 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:46:43.955245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0067f22-6723-4ec6-af85-7b826eae81aa None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:43.999003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:44.009724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:44.016917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576300-c3fb-4eb6-9875-64fe2d94419b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:44.055219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80648b8b-9048-4a1c-a545-fd79317f2f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:44.055852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80648b8b-9048-4a1c-a545-fd79317f2f91 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:44.056110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80648b8b-9048-4a1c-a545-fd79317f2f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:44.056380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80648b8b-9048-4a1c-a545-fd79317f2f91 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:44.057793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80648b8b-9048-4a1c-a545-fd79317f2f91 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:44.058070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80648b8b-9048-4a1c-a545-fd79317f2f91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:44.059564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80648b8b-9048-4a1c-a545-fd79317f2f91 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:44.072284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0067f22-6723-4ec6-af85-7b826eae81aa None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:44.080522 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 252/520] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Sep 16 07:46:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:44.081284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80648b8b-9048-4a1c-a545-fd79317f2f91 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:44.116123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ddae6-b22c-45f0-a161-ce225d14b697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:44.116942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ddae6-b22c-45f0-a161-ce225d14b697 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:44.117225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ddae6-b22c-45f0-a161-ce225d14b697 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:44.118343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ddae6-b22c-45f0-a161-ce225d14b697 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:44.128347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ddae6-b22c-45f0-a161-ce225d14b697 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:44.129806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ddae6-b22c-45f0-a161-ce225d14b697 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-152ddae6-b22c-45f0-a161-ce225d14b697 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:44.142744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:44.143620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:44.144937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:44.144937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:44.144937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:44.144937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:44.144937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:44.144937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:44.144937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:44.144937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:44.144937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 269/521] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:44.144937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ddae6-b22c-45f0-a161-ce225d14b697 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:44.192744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80648b8b-9048-4a1c-a545-fd79317f2f91 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:44.226705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c173294-f8f1-47e5-8125-4997875ed797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:44.227428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c173294-f8f1-47e5-8125-4997875ed797 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:44.227806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c173294-f8f1-47e5-8125-4997875ed797 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:44.228185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c173294-f8f1-47e5-8125-4997875ed797 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:44.235983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c173294-f8f1-47e5-8125-4997875ed797 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:44.237504 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c173294-f8f1-47e5-8125-4997875ed797 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7c173294-f8f1-47e5-8125-4997875ed797 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:44.250250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:44.251353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:44.252428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:44.252428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:44.252428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:44.252428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:44.252428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:44.252428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:44.252428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:44.252428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:44.252428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 07:46:44.252428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:44.253318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 253/522] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:44.254147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c173294-f8f1-47e5-8125-4997875ed797 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:44.257619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:44.260697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ddae6-b22c-45f0-a161-ce225d14b697 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:44.268154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:44.295961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73e26c-7d7b-45d9-ac8f-afe34404657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:44.296701 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73e26c-7d7b-45d9-ac8f-afe34404657f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:44.297100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73e26c-7d7b-45d9-ac8f-afe34404657f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:44.297472 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73e26c-7d7b-45d9-ac8f-afe34404657f None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:44.298307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73e26c-7d7b-45d9-ac8f-afe34404657f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:44.298726 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73e26c-7d7b-45d9-ac8f-afe34404657f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:44.303869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73e26c-7d7b-45d9-ac8f-afe34404657f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:44.313508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 270/523] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:44.314350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73e26c-7d7b-45d9-ac8f-afe34404657f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:44.360694 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c173294-f8f1-47e5-8125-4997875ed797 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:44.395862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13689e5-9578-4da8-8783-03f56a582418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:44.396510 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13689e5-9578-4da8-8783-03f56a582418 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:44.396784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13689e5-9578-4da8-8783-03f56a582418 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:44.397051 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13689e5-9578-4da8-8783-03f56a582418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:44.423322 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73e26c-7d7b-45d9-ac8f-afe34404657f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:44.460154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8bcd9-705a-415d-8d09-74623eca2672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:44.460768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8bcd9-705a-415d-8d09-74623eca2672 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:44.461056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8bcd9-705a-415d-8d09-74623eca2672 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:44.461329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8bcd9-705a-415d-8d09-74623eca2672 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:44.462025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8bcd9-705a-415d-8d09-74623eca2672 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:44.462324 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8bcd9-705a-415d-8d09-74623eca2672 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:44.464325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8bcd9-705a-415d-8d09-74623eca2672 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:44.472874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13689e5-9578-4da8-8783-03f56a582418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:44.474808 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13689e5-9578-4da8-8783-03f56a582418 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:44.478504 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:44.487224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 271/524] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:44.487224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8bcd9-705a-415d-8d09-74623eca2672 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:44.514792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:44.587381 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 254/525] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:44.587381 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13689e5-9578-4da8-8783-03f56a582418 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:44.597073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8bcd9-705a-415d-8d09-74623eca2672 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:44.632281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd676672-4a96-4830-8b1b-f33437b8e903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:44.633054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd676672-4a96-4830-8b1b-f33437b8e903 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:44.633341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd676672-4a96-4830-8b1b-f33437b8e903 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:44.633609 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd676672-4a96-4830-8b1b-f33437b8e903 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:44.634786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd676672-4a96-4830-8b1b-f33437b8e903 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:44.635638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd676672-4a96-4830-8b1b-f33437b8e903 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:44.637068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd676672-4a96-4830-8b1b-f33437b8e903 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:44.655307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 272/526] 10.222.0.59 () {56 vars in 1139 bytes} [Fri Sep 16 07:46:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:44.656135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd676672-4a96-4830-8b1b-f33437b8e903 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:44.698306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13689e5-9578-4da8-8783-03f56a582418 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:44.701575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:44.735418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834d083-844a-43fc-91be-bf842d3fa54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:44.735982 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834d083-844a-43fc-91be-bf842d3fa54a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:44.736264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834d083-844a-43fc-91be-bf842d3fa54a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:44.736537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834d083-844a-43fc-91be-bf842d3fa54a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:44.747363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834d083-844a-43fc-91be-bf842d3fa54a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:44.748822 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834d083-844a-43fc-91be-bf842d3fa54a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:44.760435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f834d083-844a-43fc-91be-bf842d3fa54a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:44.761572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:44.762447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:44.763414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:44.763414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:44.763414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:44.763414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:44.763414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:44.763414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:44.763414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:44.763414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:44.763414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 255/527] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:44.764058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834d083-844a-43fc-91be-bf842d3fa54a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:44.779638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd676672-4a96-4830-8b1b-f33437b8e903 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:44.818443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d69a79-81f3-471c-9bd7-f2b2e4892a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:44.819079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d69a79-81f3-471c-9bd7-f2b2e4892a65 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:44.819360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d69a79-81f3-471c-9bd7-f2b2e4892a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:44.819636 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d69a79-81f3-471c-9bd7-f2b2e4892a65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:44.820371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d69a79-81f3-471c-9bd7-f2b2e4892a65 None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=c214b2c97a7b41698b433eac8379442a) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:44.820665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d69a79-81f3-471c-9bd7-f2b2e4892a65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:44.822618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d69a79-81f3-471c-9bd7-f2b2e4892a65 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:44.838727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 273/528] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:44 2022] GET /identity/v3/role_assignments?user.id=c214b2c97a7b41698b433eac8379442a&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:44.838727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d69a79-81f3-471c-9bd7-f2b2e4892a65 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:44.890651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834d083-844a-43fc-91be-bf842d3fa54a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:44.940460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b79e8-6d7f-4b12-8ec1-8f01a4a65fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:44.941668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b79e8-6d7f-4b12-8ec1-8f01a4a65fba None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:44.942016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b79e8-6d7f-4b12-8ec1-8f01a4a65fba None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:44.942666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b79e8-6d7f-4b12-8ec1-8f01a4a65fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:44.957055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b79e8-6d7f-4b12-8ec1-8f01a4a65fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:44.959458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b79e8-6d7f-4b12-8ec1-8f01a4a65fba None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:44.961545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d69a79-81f3-471c-9bd7-f2b2e4892a65 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8d2b79e8-6d7f-4b12-8ec1-8f01a4a65fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:44.980000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:44.981209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:44.982260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:44.982260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:44.982260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:44.987243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 256/529] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:44.988303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b79e8-6d7f-4b12-8ec1-8f01a4a65fba None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:45.004450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e17eb6-656e-48ca-8f7e-3526e8866b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:45.005261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e17eb6-656e-48ca-8f7e-3526e8866b49 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:45.005798 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e17eb6-656e-48ca-8f7e-3526e8866b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:45.006078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e17eb6-656e-48ca-8f7e-3526e8866b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:45.011887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:45.031966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:45.088017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e17eb6-656e-48ca-8f7e-3526e8866b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:45.089642 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e17eb6-656e-48ca-8f7e-3526e8866b49 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:45.107432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b79e8-6d7f-4b12-8ec1-8f01a4a65fba None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:45.152337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c137a-0703-43b2-ace1-d02e8d6b92a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:45.153194 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c137a-0703-43b2-ace1-d02e8d6b92a6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:45.153655 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c137a-0703-43b2-ace1-d02e8d6b92a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:45.154075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c137a-0703-43b2-ace1-d02e8d6b92a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:45.155488 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c137a-0703-43b2-ace1-d02e8d6b92a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:45.155948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c137a-0703-43b2-ace1-d02e8d6b92a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:45.158089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c137a-0703-43b2-ace1-d02e8d6b92a6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:45.197472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 257/530] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:45.198584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c137a-0703-43b2-ace1-d02e8d6b92a6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:45.222779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 274/531] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:45.224123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e17eb6-656e-48ca-8f7e-3526e8866b49 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:45.287900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:45.303425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:45.331645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c137a-0703-43b2-ace1-d02e8d6b92a6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:45.346581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e17eb6-656e-48ca-8f7e-3526e8866b49 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:45.374896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435b957-98f2-4b12-b7d0-afb5703438f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:45.375835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435b957-98f2-4b12-b7d0-afb5703438f5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:45.376332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435b957-98f2-4b12-b7d0-afb5703438f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:45.377022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435b957-98f2-4b12-b7d0-afb5703438f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:45.378058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435b957-98f2-4b12-b7d0-afb5703438f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:45.378468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435b957-98f2-4b12-b7d0-afb5703438f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:45.380161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435b957-98f2-4b12-b7d0-afb5703438f5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:45.388330 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cceaf48-7038-41d7-a3b8-82ab4759b3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:45.389120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceaf48-7038-41d7-a3b8-82ab4759b3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:45.389544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceaf48-7038-41d7-a3b8-82ab4759b3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:45.390009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceaf48-7038-41d7-a3b8-82ab4759b3c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:45.391050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cceaf48-7038-41d7-a3b8-82ab4759b3c5 None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=e32b5bf1a2524f1cbc432f38195e4c08) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:45.391521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cceaf48-7038-41d7-a3b8-82ab4759b3c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:45.393078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cceaf48-7038-41d7-a3b8-82ab4759b3c5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:45.398558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 258/532] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:45.399474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435b957-98f2-4b12-b7d0-afb5703438f5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:45.403813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 275/533] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:45 2022] GET /identity/v3/role_assignments?user.id=e32b5bf1a2524f1cbc432f38195e4c08&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:45.404787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cceaf48-7038-41d7-a3b8-82ab4759b3c5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:45.616360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cceaf48-7038-41d7-a3b8-82ab4759b3c5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:45.619637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435b957-98f2-4b12-b7d0-afb5703438f5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:45.633744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:45.659284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ad122-1d09-467e-9b9a-444c9854262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:45.660075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ad122-1d09-467e-9b9a-444c9854262f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:45.660559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ad122-1d09-467e-9b9a-444c9854262f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:45.661048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ad122-1d09-467e-9b9a-444c9854262f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:45.661948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efaf1b-82ba-4e55-a0d7-2ac040623841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:45.662772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efaf1b-82ba-4e55-a0d7-2ac040623841 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:45.663212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efaf1b-82ba-4e55-a0d7-2ac040623841 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:45.663619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efaf1b-82ba-4e55-a0d7-2ac040623841 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:45.666878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:45.669708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ad122-1d09-467e-9b9a-444c9854262f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:45.671668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ad122-1d09-467e-9b9a-444c9854262f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:45.683966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efaf1b-82ba-4e55-a0d7-2ac040623841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-4b8ad122-1d09-467e-9b9a-444c9854262f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:45.689292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:45.690121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:45.691046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:45.691046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:45.691046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:45.691046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:45.691046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:45.691046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:45.691046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:45.691046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:45.691530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efaf1b-82ba-4e55-a0d7-2ac040623841 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:45.691866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 276/534] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:45.693824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ad122-1d09-467e-9b9a-444c9854262f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-69efaf1b-82ba-4e55-a0d7-2ac040623841 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:45.705203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:45.706355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:45.707511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:45.707511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:45.707511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:45.709364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 259/535] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:45.716611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efaf1b-82ba-4e55-a0d7-2ac040623841 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:45.810157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ad122-1d09-467e-9b9a-444c9854262f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:45.848424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46fbb0-e67e-4b4b-98fc-37297877a3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:45.849378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46fbb0-e67e-4b4b-98fc-37297877a3ac None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:45.849762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46fbb0-e67e-4b4b-98fc-37297877a3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:45.850140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46fbb0-e67e-4b4b-98fc-37297877a3ac None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:45.851729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46fbb0-e67e-4b4b-98fc-37297877a3ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:45.852411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46fbb0-e67e-4b4b-98fc-37297877a3ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:45.854015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46fbb0-e67e-4b4b-98fc-37297877a3ac None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:45.860733 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efaf1b-82ba-4e55-a0d7-2ac040623841 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:45.872893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 277/536] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:45.874257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46fbb0-e67e-4b4b-98fc-37297877a3ac None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:45.912031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:45.922883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffb1f3-7dca-483b-8830-03957928d3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:45.923749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffb1f3-7dca-483b-8830-03957928d3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:45.924198 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffb1f3-7dca-483b-8830-03957928d3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:45.924601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffb1f3-7dca-483b-8830-03957928d3e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:45.925391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffb1f3-7dca-483b-8830-03957928d3e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:45.925904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffb1f3-7dca-483b-8830-03957928d3e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:45.927579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffb1f3-7dca-483b-8830-03957928d3e7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:45.946726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 260/537] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:45.954062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffb1f3-7dca-483b-8830-03957928d3e7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:45.989504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46fbb0-e67e-4b4b-98fc-37297877a3ac None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:46.031284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470f472-8c14-4887-92c4-710ec56fcdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:46.031900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470f472-8c14-4887-92c4-710ec56fcdad None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:46.032182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470f472-8c14-4887-92c4-710ec56fcdad None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:46.032455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470f472-8c14-4887-92c4-710ec56fcdad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:46.033187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470f472-8c14-4887-92c4-710ec56fcdad None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=fb514134abe54e5ca8125e572a2ff928) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:46.033498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470f472-8c14-4887-92c4-710ec56fcdad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:46.038918 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470f472-8c14-4887-92c4-710ec56fcdad None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:46.052861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 278/538] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:45 2022] GET /identity/v3/role_assignments?user.id=fb514134abe54e5ca8125e572a2ff928&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:46.059566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:46.150795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffb1f3-7dca-483b-8830-03957928d3e7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:46.179363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72144654-7a95-4993-9969-9ea820d6920e None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:46.179671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72144654-7a95-4993-9969-9ea820d6920e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:46.179911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72144654-7a95-4993-9969-9ea820d6920e None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:46.181490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 279/539] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:46:46.187594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11c431-5276-497f-9426-2091fbf28fa4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:46.188156 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11c431-5276-497f-9426-2091fbf28fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:46.188395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11c431-5276-497f-9426-2091fbf28fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:46.206491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760d83a-db4d-419e-9885-858163705767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:46.207184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760d83a-db4d-419e-9885-858163705767 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:46.207466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760d83a-db4d-419e-9885-858163705767 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:46.207742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760d83a-db4d-419e-9885-858163705767 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:46.215988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760d83a-db4d-419e-9885-858163705767 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:46.217490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760d83a-db4d-419e-9885-858163705767 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:46.227850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8d11c431-5276-497f-9426-2091fbf28fa4 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:46.229119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8d11c431-5276-497f-9426-2091fbf28fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d760d83a-db4d-419e-9885-858163705767 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:46.230172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:46.231024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:46.231877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:46.231877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:46.231877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:46.231877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:46.231877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:46.231877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:46.231877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:46.231877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:46.232284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 261/540] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:46.239721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760d83a-db4d-419e-9885-858163705767 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:46.353283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 280/541] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:46:46.426707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760d83a-db4d-419e-9885-858163705767 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:46.445300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:46.476238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4024e-9c2a-40fe-84ab-dea84973e5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:46.476854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4024e-9c2a-40fe-84ab-dea84973e5ba None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:46.477132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4024e-9c2a-40fe-84ab-dea84973e5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:46.477400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4024e-9c2a-40fe-84ab-dea84973e5ba None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:46.478094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4024e-9c2a-40fe-84ab-dea84973e5ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:46.478397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4024e-9c2a-40fe-84ab-dea84973e5ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:46.480497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4024e-9c2a-40fe-84ab-dea84973e5ba None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:46.495573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 262/542] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:46.501468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11c431-5276-497f-9426-2091fbf28fa4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:46.552297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:46.641520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11c431-5276-497f-9426-2091fbf28fa4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:46.676107 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d7fdd-905d-4d14-bfca-1ffa82cab0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:46.676778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d7fdd-905d-4d14-bfca-1ffa82cab0ba None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:46.677004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d7fdd-905d-4d14-bfca-1ffa82cab0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:46.677281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d7fdd-905d-4d14-bfca-1ffa82cab0ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:46.678097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d7fdd-905d-4d14-bfca-1ffa82cab0ba None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=a29c4e53ac9a438e9701a70f3de3f32e) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:46.678415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d7fdd-905d-4d14-bfca-1ffa82cab0ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:46.678907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4024e-9c2a-40fe-84ab-dea84973e5ba None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:46.680010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d7fdd-905d-4d14-bfca-1ffa82cab0ba None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:46.691009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 281/543] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:46 2022] GET /identity/v3/role_assignments?user.id=a29c4e53ac9a438e9701a70f3de3f32e&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:46.809736 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4024e-9c2a-40fe-84ab-dea84973e5ba None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:46.851524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdcb52-9921-413d-833e-0a974d711900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:46.852167 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdcb52-9921-413d-833e-0a974d711900 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:46.852447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdcb52-9921-413d-833e-0a974d711900 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:46.852720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdcb52-9921-413d-833e-0a974d711900 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:46.864337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bdcb52-9921-413d-833e-0a974d711900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-49bdcb52-9921-413d-833e-0a974d711900 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:46.869290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:46.870150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:46.870981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 263/544] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:46.880005 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d7fdd-905d-4d14-bfca-1ffa82cab0ba None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:46.994447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:47.003935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d7fdd-905d-4d14-bfca-1ffa82cab0ba None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:47.040808 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d319b0-3d7d-4013-b178-63b38a832d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:47.041997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d319b0-3d7d-4013-b178-63b38a832d64 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:47.042276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d319b0-3d7d-4013-b178-63b38a832d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:47.042545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d319b0-3d7d-4013-b178-63b38a832d64 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:47.043299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d319b0-3d7d-4013-b178-63b38a832d64 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:47.043645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d319b0-3d7d-4013-b178-63b38a832d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:47.045020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d319b0-3d7d-4013-b178-63b38a832d64 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:47.053724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 282/545] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:47.059104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdcb52-9921-413d-833e-0a974d711900 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:47.105934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:47.205377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdcb52-9921-413d-833e-0a974d711900 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:47.256822 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4bc650-8b09-4686-b94b-ec8b871569b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:47.257386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bc650-8b09-4686-b94b-ec8b871569b1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:47.257668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bc650-8b09-4686-b94b-ec8b871569b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:47.257940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bc650-8b09-4686-b94b-ec8b871569b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:47.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4bc650-8b09-4686-b94b-ec8b871569b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:47.270700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4bc650-8b09-4686-b94b-ec8b871569b1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1a4bc650-8b09-4686-b94b-ec8b871569b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:47.286275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:47.287230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:47.288162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:47.288162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:47.288162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:47.288162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 264/546] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:47.295942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d319b0-3d7d-4013-b178-63b38a832d64 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:47.451483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d319b0-3d7d-4013-b178-63b38a832d64 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:47.495826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa5049-df58-4597-a2e8-fe8d9e229cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:47.496448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa5049-df58-4597-a2e8-fe8d9e229cbe None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:47.496733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa5049-df58-4597-a2e8-fe8d9e229cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:47.497011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa5049-df58-4597-a2e8-fe8d9e229cbe None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:47.497656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa5049-df58-4597-a2e8-fe8d9e229cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:47.497963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa5049-df58-4597-a2e8-fe8d9e229cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:47.499448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa5049-df58-4597-a2e8-fe8d9e229cbe None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:47.507186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafaeab-4ec3-4c0d-8244-5bb0f5091d14 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:47.507422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafaeab-4ec3-4c0d-8244-5bb0f5091d14 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:47.507684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafaeab-4ec3-4c0d-8244-5bb0f5091d14 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:47.509256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 265/547] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:47.514567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a706d62-ebf2-472e-8bd4-a676e3c1f90a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:47.514881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a706d62-ebf2-472e-8bd4-a676e3c1f90a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:47.515162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a706d62-ebf2-472e-8bd4-a676e3c1f90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:47.521172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 283/548] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:47.528630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa5049-df58-4597-a2e8-fe8d9e229cbe None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:47.539611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7a706d62-ebf2-472e-8bd4-a676e3c1f90a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:47.540826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7a706d62-ebf2-472e-8bd4-a676e3c1f90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:47.551609 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:47.664330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 266/549] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:46:47.706492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa5049-df58-4597-a2e8-fe8d9e229cbe None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:47.774710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e69cd4-1ae2-416b-b343-8252edebc7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:47.775908 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e69cd4-1ae2-416b-b343-8252edebc7da None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:47.776193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e69cd4-1ae2-416b-b343-8252edebc7da None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:47.776468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e69cd4-1ae2-416b-b343-8252edebc7da None admin] PATH_INFO: `/v3/users/swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:47.787348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e69cd4-1ae2-416b-b343-8252edebc7da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:47.788818 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e69cd4-1ae2-416b-b343-8252edebc7da None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-12e69cd4-1ae2-416b-b343-8252edebc7da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:47.802797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:47.803754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:47.804672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:47.804672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:47.804672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:47.804672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:47.804672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:47.804672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:47.804672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:47.804672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:47.804672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:46:47.804672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:47.804672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 284/550] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:46:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:47.811867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a706d62-ebf2-472e-8bd4-a676e3c1f90a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:47.832710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:47.971111 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a706d62-ebf2-472e-8bd4-a676e3c1f90a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:48.006282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92931be9-2782-41e3-b95f-e9f56b9efdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:48.006925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92931be9-2782-41e3-b95f-e9f56b9efdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.007208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92931be9-2782-41e3-b95f-e9f56b9efdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.007486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92931be9-2782-41e3-b95f-e9f56b9efdf1 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:48.008222 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92931be9-2782-41e3-b95f-e9f56b9efdf1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:48.008584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92931be9-2782-41e3-b95f-e9f56b9efdf1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:48.010553 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92931be9-2782-41e3-b95f-e9f56b9efdf1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:48.024495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e69cd4-1ae2-416b-b343-8252edebc7da None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:48.033253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 267/551] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Sep 16 07:46:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:48.035659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92931be9-2782-41e3-b95f-e9f56b9efdf1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:48.151201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:48.160918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:48.194063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e69cd4-1ae2-416b-b343-8252edebc7da None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:48.219122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92931be9-2782-41e3-b95f-e9f56b9efdf1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:48.269530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bc22c-8433-4081-b639-d6c7982a61f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:48.270165 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bc22c-8433-4081-b639-d6c7982a61f0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.270441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bc22c-8433-4081-b639-d6c7982a61f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.270729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bc22c-8433-4081-b639-d6c7982a61f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:48.289475 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f53814-b0c6-42b5-8897-473de530ab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:48.290006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f53814-b0c6-42b5-8897-473de530ab2a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.290282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f53814-b0c6-42b5-8897-473de530ab2a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.290547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f53814-b0c6-42b5-8897-473de530ab2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:48.292250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bc22c-8433-4081-b639-d6c7982a61f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-c40bc22c-8433-4081-b639-d6c7982a61f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:48.297567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:48.298511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:48.299390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:48.307719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 285/552] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:48.308624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bc22c-8433-4081-b639-d6c7982a61f0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:48.380665 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f53814-b0c6-42b5-8897-473de530ab2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:48.383078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f53814-b0c6-42b5-8897-473de530ab2a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:48.439601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bc22c-8433-4081-b639-d6c7982a61f0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:48.460884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:48.479176 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:48.484967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000e00b-a9be-4af2-a0c6-05bd09e8bb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:48.486032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000e00b-a9be-4af2-a0c6-05bd09e8bb30 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.486314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000e00b-a9be-4af2-a0c6-05bd09e8bb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.486915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000e00b-a9be-4af2-a0c6-05bd09e8bb30 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:48.487577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000e00b-a9be-4af2-a0c6-05bd09e8bb30 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:48.487883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000e00b-a9be-4af2-a0c6-05bd09e8bb30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:48.489970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000e00b-a9be-4af2-a0c6-05bd09e8bb30 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:48.499997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 268/553] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:48.501383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedff2fa-6263-4fa0-9d3f-0910bc23ceba None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.501827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedff2fa-6263-4fa0-9d3f-0910bc23ceba None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.501943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 286/554] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:48.507701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedff2fa-6263-4fa0-9d3f-0910bc23ceba None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:48.507701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 269/555] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:48.507872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000e00b-a9be-4af2-a0c6-05bd09e8bb30 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:48.510783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741cb7f-bcd2-4f67-b4b5-1f46a9415c1f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.510783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741cb7f-bcd2-4f67-b4b5-1f46a9415c1f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.510783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741cb7f-bcd2-4f67-b4b5-1f46a9415c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:48.529708 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a741cb7f-bcd2-4f67-b4b5-1f46a9415c1f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:48.531391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a741cb7f-bcd2-4f67-b4b5-1f46a9415c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:48.621114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000e00b-a9be-4af2-a0c6-05bd09e8bb30 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:48.623060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 270/556] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:48.623963 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741cb7f-bcd2-4f67-b4b5-1f46a9415c1f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:48.657390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d2988-e428-4276-b022-2f6648d581ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:48.658368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d2988-e428-4276-b022-2f6648d581ac None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.658923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d2988-e428-4276-b022-2f6648d581ac None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.659457 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d2988-e428-4276-b022-2f6648d581ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:48.671011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d2988-e428-4276-b022-2f6648d581ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:48.672753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d2988-e428-4276-b022-2f6648d581ac None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-cd4d2988-e428-4276-b022-2f6648d581ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:48.689874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:48.690970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:48.691885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:48.691885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:48.691885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:48.693985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 287/557] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:48.699730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d2988-e428-4276-b022-2f6648d581ac None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:48.752553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:48.754924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:48.758853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741cb7f-bcd2-4f67-b4b5-1f46a9415c1f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:48.794904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912c08f-742f-4277-b818-fb4f7102f39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:48.795551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912c08f-742f-4277-b818-fb4f7102f39c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.795892 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912c08f-742f-4277-b818-fb4f7102f39c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.796215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912c08f-742f-4277-b818-fb4f7102f39c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:48.811877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d2988-e428-4276-b022-2f6648d581ac None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:48.812058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912c08f-742f-4277-b818-fb4f7102f39c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:48.813622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e912c08f-742f-4277-b818-fb4f7102f39c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-e912c08f-742f-4277-b818-fb4f7102f39c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:48.832872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:48.834512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:48.835638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:48.835638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:48.835638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:48.841865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 271/558] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:48.841865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3ed68-d489-45cd-8393-2c42a50a1e8e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.841865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3ed68-d489-45cd-8393-2c42a50a1e8e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.841865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3ed68-d489-45cd-8393-2c42a50a1e8e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:48.845523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 272/559] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:48.846301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3ed68-d489-45cd-8393-2c42a50a1e8e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:48.861478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279b470-97d1-46aa-9847-52a2c325091f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:48.862537 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279b470-97d1-46aa-9847-52a2c325091f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.863065 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279b470-97d1-46aa-9847-52a2c325091f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.863564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279b470-97d1-46aa-9847-52a2c325091f None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:48.864425 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279b470-97d1-46aa-9847-52a2c325091f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:48.865060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279b470-97d1-46aa-9847-52a2c325091f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:48.866647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279b470-97d1-46aa-9847-52a2c325091f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:48.883491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 288/560] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:48.884615 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7005-000d-42a8-a458-ce0274bcec2a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.884979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7005-000d-42a8-a458-ce0274bcec2a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.885336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7005-000d-42a8-a458-ce0274bcec2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:48.907971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2f8a7005-000d-42a8-a458-ce0274bcec2a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:48.909415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2f8a7005-000d-42a8-a458-ce0274bcec2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:48.964394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3ed68-d489-45cd-8393-2c42a50a1e8e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:48.998560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec9162-74e2-40aa-99ce-944a4fca58e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:48.999189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec9162-74e2-40aa-99ce-944a4fca58e6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:48.999471 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec9162-74e2-40aa-99ce-944a4fca58e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:48.999746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec9162-74e2-40aa-99ce-944a4fca58e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.000385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec9162-74e2-40aa-99ce-944a4fca58e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:49.000689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec9162-74e2-40aa-99ce-944a4fca58e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:49.002057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec9162-74e2-40aa-99ce-944a4fca58e6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:49.017948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 273/561] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:49.017948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec9162-74e2-40aa-99ce-944a4fca58e6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.027961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:49.042724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 289/562] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:49.042724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a7005-000d-42a8-a458-ce0274bcec2a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.080129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:49.134407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec9162-74e2-40aa-99ce-944a4fca58e6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:49.171756 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47564cf-6783-46fa-bc99-cf4ee5ba518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:49.172401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47564cf-6783-46fa-bc99-cf4ee5ba518f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.172685 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47564cf-6783-46fa-bc99-cf4ee5ba518f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.172795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a7005-000d-42a8-a458-ce0274bcec2a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:49.172957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47564cf-6783-46fa-bc99-cf4ee5ba518f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.180470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47564cf-6783-46fa-bc99-cf4ee5ba518f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:49.182144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47564cf-6783-46fa-bc99-cf4ee5ba518f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a47564cf-6783-46fa-bc99-cf4ee5ba518f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:49.195117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:49.196237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:49.197231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:49.197231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:49.197231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:49.197231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:49.197231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:49.197231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:49.197231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:49.197231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:49.197231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 16 07:46:49.197231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:49.197231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 274/563] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:49.197932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47564cf-6783-46fa-bc99-cf4ee5ba518f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.223427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03797a41-f46c-4dac-aad7-f79fb8ae63e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:49.224245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03797a41-f46c-4dac-aad7-f79fb8ae63e9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.224648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03797a41-f46c-4dac-aad7-f79fb8ae63e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.225014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03797a41-f46c-4dac-aad7-f79fb8ae63e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.237505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03797a41-f46c-4dac-aad7-f79fb8ae63e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-03797a41-f46c-4dac-aad7-f79fb8ae63e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:49.242448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:49.244320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 290/564] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:49.246282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03797a41-f46c-4dac-aad7-f79fb8ae63e9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.292733 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:49.326144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47564cf-6783-46fa-bc99-cf4ee5ba518f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:49.351771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:49.357648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03797a41-f46c-4dac-aad7-f79fb8ae63e9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:49.362436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df739c13-1321-479b-aed5-8aace729a544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:49.363128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df739c13-1321-479b-aed5-8aace729a544 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.363408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df739c13-1321-479b-aed5-8aace729a544 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.363681 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df739c13-1321-479b-aed5-8aace729a544 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.370487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df739c13-1321-479b-aed5-8aace729a544 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:49.371993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df739c13-1321-479b-aed5-8aace729a544 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-df739c13-1321-479b-aed5-8aace729a544 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:49.383065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:49.383965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:49.385420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:49.385420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:49.385420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:49.385420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:49.385420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:49.385420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:49.385420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:49.385420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:49.385420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 275/565] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:49.385420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df739c13-1321-479b-aed5-8aace729a544 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.396656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab005a74-bbc8-4162-b0ca-cdf5c6ab7ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:49.397266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab005a74-bbc8-4162-b0ca-cdf5c6ab7ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.397541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab005a74-bbc8-4162-b0ca-cdf5c6ab7ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.397814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab005a74-bbc8-4162-b0ca-cdf5c6ab7ce1 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.398536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab005a74-bbc8-4162-b0ca-cdf5c6ab7ce1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:49.398867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab005a74-bbc8-4162-b0ca-cdf5c6ab7ce1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:49.400961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab005a74-bbc8-4162-b0ca-cdf5c6ab7ce1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:49.417906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 291/566] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:49.419093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8bc53-1c81-4d54-8d86-84a163920eb1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.419375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8bc53-1c81-4d54-8d86-84a163920eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.419647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8bc53-1c81-4d54-8d86-84a163920eb1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.420826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 292/567] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:49.421512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8bc53-1c81-4d54-8d86-84a163920eb1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.486972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df739c13-1321-479b-aed5-8aace729a544 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:49.513913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:49.519649 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bfb58-3196-4205-9b14-229d34a83c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:49.520241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bfb58-3196-4205-9b14-229d34a83c8a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.520519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bfb58-3196-4205-9b14-229d34a83c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.520789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bfb58-3196-4205-9b14-229d34a83c8a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.521439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bfb58-3196-4205-9b14-229d34a83c8a None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:49.522903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bfb58-3196-4205-9b14-229d34a83c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:49.523200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bfb58-3196-4205-9b14-229d34a83c8a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:49.532034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 276/568] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:49.532630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bfb58-3196-4205-9b14-229d34a83c8a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.538351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8bc53-1c81-4d54-8d86-84a163920eb1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:49.571286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01768d1d-92e5-4240-b9d4-16849bd77687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:49.571908 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01768d1d-92e5-4240-b9d4-16849bd77687 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.572700 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01768d1d-92e5-4240-b9d4-16849bd77687 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.572970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01768d1d-92e5-4240-b9d4-16849bd77687 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.582952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01768d1d-92e5-4240-b9d4-16849bd77687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-01768d1d-92e5-4240-b9d4-16849bd77687 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:49.587678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:49.589154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 293/569] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:49.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01768d1d-92e5-4240-b9d4-16849bd77687 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.605640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:49.634378 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bfb58-3196-4205-9b14-229d34a83c8a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:49.667969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbfed0-a0b6-4e50-8e87-349b1c2b2514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:49.668600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbfed0-a0b6-4e50-8e87-349b1c2b2514 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.668765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbfed0-a0b6-4e50-8e87-349b1c2b2514 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.669036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbfed0-a0b6-4e50-8e87-349b1c2b2514 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.669726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbfed0-a0b6-4e50-8e87-349b1c2b2514 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:49.670035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbfed0-a0b6-4e50-8e87-349b1c2b2514 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:49.671588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbfed0-a0b6-4e50-8e87-349b1c2b2514 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:49.683762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 277/570] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:49.685066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f65ad-44e7-415f-9c86-bfbc6fd2822d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.685353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f65ad-44e7-415f-9c86-bfbc6fd2822d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.685625 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f65ad-44e7-415f-9c86-bfbc6fd2822d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.693705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01768d1d-92e5-4240-b9d4-16849bd77687 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:49.704004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fe6f65ad-44e7-415f-9c86-bfbc6fd2822d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:49.705228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fe6f65ad-44e7-415f-9c86-bfbc6fd2822d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:49.728804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd1a6a-6de3-4183-951f-41077b90b825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:49.729432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd1a6a-6de3-4183-951f-41077b90b825 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.729707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd1a6a-6de3-4183-951f-41077b90b825 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.729993 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd1a6a-6de3-4183-951f-41077b90b825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.741648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd1a6a-6de3-4183-951f-41077b90b825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:49.743149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd1a6a-6de3-4183-951f-41077b90b825 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:49.753216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-ecfd1a6a-6de3-4183-951f-41077b90b825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:49.758366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:49.760288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:49.761280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:49.761280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:49.761280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:49.761280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 294/571] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:49.761951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd1a6a-6de3-4183-951f-41077b90b825 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.773998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 278/572] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:49.774856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f65ad-44e7-415f-9c86-bfbc6fd2822d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.843333 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:49.876814 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f65ad-44e7-415f-9c86-bfbc6fd2822d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:49.880517 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd1a6a-6de3-4183-951f-41077b90b825 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:49.911473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c709e-d1a3-4858-9b85-9250a70b566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:49.912176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c709e-d1a3-4858-9b85-9250a70b566a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.912399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c709e-d1a3-4858-9b85-9250a70b566a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.912659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c709e-d1a3-4858-9b85-9250a70b566a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.913313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c709e-d1a3-4858-9b85-9250a70b566a None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:49.913616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c709e-d1a3-4858-9b85-9250a70b566a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:49.915006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c709e-d1a3-4858-9b85-9250a70b566a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:49.915261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea064aa-6d5f-4bf1-9383-5e54aee78b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:49.919961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea064aa-6d5f-4bf1-9383-5e54aee78b07 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:49.919961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea064aa-6d5f-4bf1-9383-5e54aee78b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:49.919961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea064aa-6d5f-4bf1-9383-5e54aee78b07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:49.927637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 279/573] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:49.927637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c709e-d1a3-4858-9b85-9250a70b566a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.933017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea064aa-6d5f-4bf1-9383-5e54aee78b07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:49.934535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea064aa-6d5f-4bf1-9383-5e54aee78b07 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-7ea064aa-6d5f-4bf1-9383-5e54aee78b07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:49.949646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:49.950589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:49.951505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:49.951505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:49.951505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:49.951505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 295/574] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:49.951505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea064aa-6d5f-4bf1-9383-5e54aee78b07 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:49.981189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:50.034973 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c709e-d1a3-4858-9b85-9250a70b566a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.058352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea064aa-6d5f-4bf1-9383-5e54aee78b07 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.068680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47338-79e6-4f62-a75b-91b404b55a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.069281 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47338-79e6-4f62-a75b-91b404b55a78 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.069558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47338-79e6-4f62-a75b-91b404b55a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.069825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47338-79e6-4f62-a75b-91b404b55a78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.070535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc47338-79e6-4f62-a75b-91b404b55a78 None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=a1ae39fdd6ff4841843c9826dd88229a) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:50.070883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc47338-79e6-4f62-a75b-91b404b55a78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:50.072339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc47338-79e6-4f62-a75b-91b404b55a78 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:50.075485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:50.077650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 280/575] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/role_assignments?user.id=a1ae39fdd6ff4841843c9826dd88229a&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:50.078450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47338-79e6-4f62-a75b-91b404b55a78 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:50.093135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324dae5f-61aa-4525-9764-b4046f3ecf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.094130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324dae5f-61aa-4525-9764-b4046f3ecf6e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.094590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324dae5f-61aa-4525-9764-b4046f3ecf6e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.095079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324dae5f-61aa-4525-9764-b4046f3ecf6e None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.095856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324dae5f-61aa-4525-9764-b4046f3ecf6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:50.096162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324dae5f-61aa-4525-9764-b4046f3ecf6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:50.097583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324dae5f-61aa-4525-9764-b4046f3ecf6e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:50.116693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 296/576] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:50.116693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324dae5f-61aa-4525-9764-b4046f3ecf6e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:50.205038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47338-79e6-4f62-a75b-91b404b55a78 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.215989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:50.232459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324dae5f-61aa-4525-9764-b4046f3ecf6e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.243446 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62ec03-96ab-4f2b-8c7b-2e7646014b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.244697 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62ec03-96ab-4f2b-8c7b-2e7646014b03 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.244977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62ec03-96ab-4f2b-8c7b-2e7646014b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.245250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62ec03-96ab-4f2b-8c7b-2e7646014b03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.257328 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62ec03-96ab-4f2b-8c7b-2e7646014b03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:50.258834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62ec03-96ab-4f2b-8c7b-2e7646014b03 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:50.270430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ea330-b2bb-445b-99c5-70a9a2b5cd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.271060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ea330-b2bb-445b-99c5-70a9a2b5cd30 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.271338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ea330-b2bb-445b-99c5-70a9a2b5cd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.271610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ea330-b2bb-445b-99c5-70a9a2b5cd30 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.272244 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ea330-b2bb-445b-99c5-70a9a2b5cd30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:50.272547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ea330-b2bb-445b-99c5-70a9a2b5cd30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-4e62ec03-96ab-4f2b-8c7b-2e7646014b03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:50.273455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:50.274358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:50.276161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:50.276161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:50.276161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:50.276350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ea330-b2bb-445b-99c5-70a9a2b5cd30 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:50.279334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 281/577] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:50.279958 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62ec03-96ab-4f2b-8c7b-2e7646014b03 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:50.291771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 297/578] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:50.291771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ea330-b2bb-445b-99c5-70a9a2b5cd30 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:50.334907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:50.388158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62ec03-96ab-4f2b-8c7b-2e7646014b03 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.402700 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ea330-b2bb-445b-99c5-70a9a2b5cd30 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.422746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e81e8-efa9-44f7-9797-50e162e3331e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.423387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e81e8-efa9-44f7-9797-50e162e3331e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.423668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e81e8-efa9-44f7-9797-50e162e3331e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.423933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e81e8-efa9-44f7-9797-50e162e3331e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.434725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e81e8-efa9-44f7-9797-50e162e3331e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:50.439012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a8657-ce19-47e0-ba0b-2c9ef5878462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.439653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a8657-ce19-47e0-ba0b-2c9ef5878462 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.439934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a8657-ce19-47e0-ba0b-2c9ef5878462 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.440207 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a8657-ce19-47e0-ba0b-2c9ef5878462 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c25e81e8-efa9-44f7-9797-50e162e3331e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:50.440301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:50.441149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:50.441976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:50.446372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 282/579] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:50.447276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e81e8-efa9-44f7-9797-50e162e3331e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:50.449822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a8657-ce19-47e0-ba0b-2c9ef5878462 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:50.451424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a8657-ce19-47e0-ba0b-2c9ef5878462 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b98a8657-ce19-47e0-ba0b-2c9ef5878462 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:50.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:50.465817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:50.467144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:50.467144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:50.467144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:50.467144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:50.467144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:50.467144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:50.467144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:50.467144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:50.467144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 298/580] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:50.467144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a8657-ce19-47e0-ba0b-2c9ef5878462 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:50.474712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:50.563102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e81e8-efa9-44f7-9797-50e162e3331e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.576395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:50.579966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a8657-ce19-47e0-ba0b-2c9ef5878462 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.598590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15718a46-8102-40bc-b47b-60e959a6f6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.599225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15718a46-8102-40bc-b47b-60e959a6f6fe None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.599498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15718a46-8102-40bc-b47b-60e959a6f6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.599766 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15718a46-8102-40bc-b47b-60e959a6f6fe None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.600412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15718a46-8102-40bc-b47b-60e959a6f6fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:50.600710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15718a46-8102-40bc-b47b-60e959a6f6fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:50.602082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15718a46-8102-40bc-b47b-60e959a6f6fe None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:50.615210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc97fbf-5aa8-456c-938d-e162d955f0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.615917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 283/581] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:50.616736 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15718a46-8102-40bc-b47b-60e959a6f6fe None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:50.620314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc97fbf-5aa8-456c-938d-e162d955f0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.620815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc97fbf-5aa8-456c-938d-e162d955f0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.621172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc97fbf-5aa8-456c-938d-e162d955f0c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.629420 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc97fbf-5aa8-456c-938d-e162d955f0c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:50.630894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc97fbf-5aa8-456c-938d-e162d955f0c8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-4cc97fbf-5aa8-456c-938d-e162d955f0c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:50.642146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:50.643158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 299/582] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Sep 16 07:46:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:50.644119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc97fbf-5aa8-456c-938d-e162d955f0c8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:50.704634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:50.724731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15718a46-8102-40bc-b47b-60e959a6f6fe None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.760654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2ce14-731f-4427-aafb-a1f40bc5ee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.761269 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2ce14-731f-4427-aafb-a1f40bc5ee14 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.761547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2ce14-731f-4427-aafb-a1f40bc5ee14 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.761820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2ce14-731f-4427-aafb-a1f40bc5ee14 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.762476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2ce14-731f-4427-aafb-a1f40bc5ee14 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:50.762815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2ce14-731f-4427-aafb-a1f40bc5ee14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:50.764191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2ce14-731f-4427-aafb-a1f40bc5ee14 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:50.764899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc97fbf-5aa8-456c-938d-e162d955f0c8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.777536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 284/583] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:50.778366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2ce14-731f-4427-aafb-a1f40bc5ee14 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:50.802174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfb2eb-fad2-4e28-b3c1-d7c30ec1fa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.802822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfb2eb-fad2-4e28-b3c1-d7c30ec1fa6c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.803120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfb2eb-fad2-4e28-b3c1-d7c30ec1fa6c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.803371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfb2eb-fad2-4e28-b3c1-d7c30ec1fa6c None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.804056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfb2eb-fad2-4e28-b3c1-d7c30ec1fa6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:50.804366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfb2eb-fad2-4e28-b3c1-d7c30ec1fa6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:50.805789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfb2eb-fad2-4e28-b3c1-d7c30ec1fa6c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:50.823841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 300/584] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:50.823841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfb2eb-fad2-4e28-b3c1-d7c30ec1fa6c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:50.842922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:50.885919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2ce14-731f-4427-aafb-a1f40bc5ee14 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.923195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db360716-8e16-4f8e-8223-4fb0432ddf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.923993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db360716-8e16-4f8e-8223-4fb0432ddf70 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.924274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db360716-8e16-4f8e-8223-4fb0432ddf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.924734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db360716-8e16-4f8e-8223-4fb0432ddf70 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.927422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfb2eb-fad2-4e28-b3c1-d7c30ec1fa6c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:50.932703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db360716-8e16-4f8e-8223-4fb0432ddf70 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:50.934242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db360716-8e16-4f8e-8223-4fb0432ddf70 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-db360716-8e16-4f8e-8223-4fb0432ddf70 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:50.946830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:50.948442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 285/585] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:50.949402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db360716-8e16-4f8e-8223-4fb0432ddf70 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:50.957431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:50.970228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc1814-9477-4505-963c-6324bc085e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:50.970864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc1814-9477-4505-963c-6324bc085e98 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:50.972365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc1814-9477-4505-963c-6324bc085e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:50.972365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc1814-9477-4505-963c-6324bc085e98 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:50.972365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc1814-9477-4505-963c-6324bc085e98 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:50.972616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc1814-9477-4505-963c-6324bc085e98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:50.974023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc1814-9477-4505-963c-6324bc085e98 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:50.991351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 301/586] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Sep 16 07:46:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:50.993413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc1814-9477-4505-963c-6324bc085e98 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:51.058375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db360716-8e16-4f8e-8223-4fb0432ddf70 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:51.079980 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:51.094022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46bc42-bcf7-40af-8a43-8ba89a430694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:51.094664 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46bc42-bcf7-40af-8a43-8ba89a430694 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:51.094987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46bc42-bcf7-40af-8a43-8ba89a430694 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:51.095267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46bc42-bcf7-40af-8a43-8ba89a430694 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:51.104875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc1814-9477-4505-963c-6324bc085e98 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:51.105896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46bc42-bcf7-40af-8a43-8ba89a430694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:51.107431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46bc42-bcf7-40af-8a43-8ba89a430694 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8b46bc42-bcf7-40af-8a43-8ba89a430694 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:51.121678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:51.122626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:51.123940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:51.123940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:51.123940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:51.127362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 286/587] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:51.128188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46bc42-bcf7-40af-8a43-8ba89a430694 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:51.142462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636dda3-a77c-4680-820b-e0219f7f9ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:51.143167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636dda3-a77c-4680-820b-e0219f7f9ac7 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:51.143453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636dda3-a77c-4680-820b-e0219f7f9ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:51.143750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636dda3-a77c-4680-820b-e0219f7f9ac7 None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/c214b2c97a7b41698b433eac8379442a/roles/691d078c15384159b1eec2b2ceaad9ed` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:51.160046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636dda3-a77c-4680-820b-e0219f7f9ac7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=c214b2c97a7b41698b433eac8379442a, role_id=691d078c15384159b1eec2b2ceaad9ed)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:51.161470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636dda3-a77c-4680-820b-e0219f7f9ac7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:51.184529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:51.226927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46bc42-bcf7-40af-8a43-8ba89a430694 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:51.233863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 302/588] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:46:50 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/c214b2c97a7b41698b433eac8379442a/roles/691d078c15384159b1eec2b2ceaad9ed => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:51.234803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636dda3-a77c-4680-820b-e0219f7f9ac7 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:51.283670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba15bbd-8151-42f2-9742-d34f6b1cf041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:51.284593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba15bbd-8151-42f2-9742-d34f6b1cf041 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:51.285075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba15bbd-8151-42f2-9742-d34f6b1cf041 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:51.285524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba15bbd-8151-42f2-9742-d34f6b1cf041 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:51.286580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba15bbd-8151-42f2-9742-d34f6b1cf041 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:51.287149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba15bbd-8151-42f2-9742-d34f6b1cf041 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:51.289621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba15bbd-8151-42f2-9742-d34f6b1cf041 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:51.317582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 287/589] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Sep 16 07:46:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:51.318278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba15bbd-8151-42f2-9742-d34f6b1cf041 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:51.374515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636dda3-a77c-4680-820b-e0219f7f9ac7 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:51.407512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:51.420982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc81c59-2564-4611-8e49-b7b24efb3eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:51.422145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc81c59-2564-4611-8e49-b7b24efb3eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:51.422145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc81c59-2564-4611-8e49-b7b24efb3eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:51.422392 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc81c59-2564-4611-8e49-b7b24efb3eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:51.434516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc81c59-2564-4611-8e49-b7b24efb3eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:51.436068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc81c59-2564-4611-8e49-b7b24efb3eb9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-7dc81c59-2564-4611-8e49-b7b24efb3eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:51.454527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:51.459776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:51.460676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:51.460676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:51.460676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:51.460676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 303/590] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:51.460676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc81c59-2564-4611-8e49-b7b24efb3eb9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:51.464504 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba15bbd-8151-42f2-9742-d34f6b1cf041 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:51.478763 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:51.502277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6152e-fb6a-43db-a9c1-343696ddb464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:51.503041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6152e-fb6a-43db-a9c1-343696ddb464 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:51.503415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6152e-fb6a-43db-a9c1-343696ddb464 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:51.503782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6152e-fb6a-43db-a9c1-343696ddb464 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:51.504630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6152e-fb6a-43db-a9c1-343696ddb464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:51.505133 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6152e-fb6a-43db-a9c1-343696ddb464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:51.506575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6152e-fb6a-43db-a9c1-343696ddb464 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:51.519900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 288/591] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:51.520860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6152e-fb6a-43db-a9c1-343696ddb464 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:51.566250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc81c59-2564-4611-8e49-b7b24efb3eb9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:51.614542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafce8ce-c96e-4cf3-b747-7c21ebc31c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:51.615495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce8ce-c96e-4cf3-b747-7c21ebc31c7d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:51.615782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce8ce-c96e-4cf3-b747-7c21ebc31c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:51.616065 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce8ce-c96e-4cf3-b747-7c21ebc31c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:51.628833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafce8ce-c96e-4cf3-b747-7c21ebc31c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:51.630527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafce8ce-c96e-4cf3-b747-7c21ebc31c7d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:51.634054 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6152e-fb6a-43db-a9c1-343696ddb464 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-fafce8ce-c96e-4cf3-b747-7c21ebc31c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:51.646202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:51.647193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:51.648097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:51.648097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:51.648097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:51.648097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 304/592] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:51.651435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafce8ce-c96e-4cf3-b747-7c21ebc31c7d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:51.679841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf427f67-3224-4c7d-ba0a-626a5faed51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:51.680696 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf427f67-3224-4c7d-ba0a-626a5faed51e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:51.681340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf427f67-3224-4c7d-ba0a-626a5faed51e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:51.682187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf427f67-3224-4c7d-ba0a-626a5faed51e None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:51.683155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf427f67-3224-4c7d-ba0a-626a5faed51e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:51.683520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf427f67-3224-4c7d-ba0a-626a5faed51e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:51.685411 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf427f67-3224-4c7d-ba0a-626a5faed51e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:51.687414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:51.702800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 289/593] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:51.704006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf427f67-3224-4c7d-ba0a-626a5faed51e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:51.750420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:51.777096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafce8ce-c96e-4cf3-b747-7c21ebc31c7d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:51.817998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6edf3b-3c40-476d-bc76-2670073b28c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:51.818569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6edf3b-3c40-476d-bc76-2670073b28c2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:51.818919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6edf3b-3c40-476d-bc76-2670073b28c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:51.819191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6edf3b-3c40-476d-bc76-2670073b28c2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:51.828664 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6edf3b-3c40-476d-bc76-2670073b28c2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:51.830026 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf427f67-3224-4c7d-ba0a-626a5faed51e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:51.830141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6edf3b-3c40-476d-bc76-2670073b28c2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-5d6edf3b-3c40-476d-bc76-2670073b28c2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:51.842998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:51.843967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:51.844899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:51.844899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:51.844899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:51.844899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:51.844899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:51.844899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:51.844899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:51.844899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:51.844899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 07:46:51.844899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:51.848939 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 305/594] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:51.850271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6edf3b-3c40-476d-bc76-2670073b28c2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:51.872208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4655bd5-5f15-4dda-8b91-24de8720adb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:51.873184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4655bd5-5f15-4dda-8b91-24de8720adb9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:51.873659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4655bd5-5f15-4dda-8b91-24de8720adb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:51.874133 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4655bd5-5f15-4dda-8b91-24de8720adb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:51.875098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4655bd5-5f15-4dda-8b91-24de8720adb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:51.875598 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4655bd5-5f15-4dda-8b91-24de8720adb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:51.877237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4655bd5-5f15-4dda-8b91-24de8720adb9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:51.891513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 290/595] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:51.893098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4655bd5-5f15-4dda-8b91-24de8720adb9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:51.968603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:51.980155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6edf3b-3c40-476d-bc76-2670073b28c2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:52.000253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:52.010091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4655bd5-5f15-4dda-8b91-24de8720adb9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:52.020788 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3db399-88fd-4f12-b0e3-29a5146d4d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:52.021441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db399-88fd-4f12-b0e3-29a5146d4d31 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.021724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db399-88fd-4f12-b0e3-29a5146d4d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.022009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db399-88fd-4f12-b0e3-29a5146d4d31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.030081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3db399-88fd-4f12-b0e3-29a5146d4d31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:52.031579 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3db399-88fd-4f12-b0e3-29a5146d4d31 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-fc3db399-88fd-4f12-b0e3-29a5146d4d31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:52.046760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:52.047708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:52.048584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:52.048584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:52.048584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:52.048584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:52.048584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:52.048584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:52.048584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:52.048584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:52.050223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2967bce-74b2-4468-bab4-b11861142a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:52.051090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2967bce-74b2-4468-bab4-b11861142a30 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.051576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2967bce-74b2-4468-bab4-b11861142a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.052049 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2967bce-74b2-4468-bab4-b11861142a30 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.052666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 306/596] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:52.053455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2967bce-74b2-4468-bab4-b11861142a30 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:52.053571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3db399-88fd-4f12-b0e3-29a5146d4d31 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:52.054242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2967bce-74b2-4468-bab4-b11861142a30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:52.055991 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2967bce-74b2-4468-bab4-b11861142a30 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:52.076445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 291/597] 10.222.0.59 () {56 vars in 1139 bytes} [Fri Sep 16 07:46:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:52.077567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2967bce-74b2-4468-bab4-b11861142a30 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:52.188729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2967bce-74b2-4468-bab4-b11861142a30 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:52.189116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3db399-88fd-4f12-b0e3-29a5146d4d31 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:52.227330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c48f4-b979-4402-a9a8-b90610bd1305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:52.228337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c48f4-b979-4402-a9a8-b90610bd1305 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.228814 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c48f4-b979-4402-a9a8-b90610bd1305 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.229424 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c48f4-b979-4402-a9a8-b90610bd1305 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.230405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c48f4-b979-4402-a9a8-b90610bd1305 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:52.230929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c48f4-b979-4402-a9a8-b90610bd1305 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:52.232570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306cf49-e1c3-44da-a41c-50a7d81462e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:52.232854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c48f4-b979-4402-a9a8-b90610bd1305 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:52.233212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306cf49-e1c3-44da-a41c-50a7d81462e4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.233496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306cf49-e1c3-44da-a41c-50a7d81462e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.233771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306cf49-e1c3-44da-a41c-50a7d81462e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.242126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306cf49-e1c3-44da-a41c-50a7d81462e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:52.244933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306cf49-e1c3-44da-a41c-50a7d81462e4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:52.246991 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 292/598] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:52.248085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c48f4-b979-4402-a9a8-b90610bd1305 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-2306cf49-e1c3-44da-a41c-50a7d81462e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:52.257309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:52.258222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:52.260257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:52.260257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:52.260257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:52.260257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:52.260257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:52.260257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:52.260257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:52.260257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:52.261306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:52.263590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 307/599] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:52.264650 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9738b4c-fdf6-4f18-84d5-586e87f01ad9 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.264942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9738b4c-fdf6-4f18-84d5-586e87f01ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.265227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9738b4c-fdf6-4f18-84d5-586e87f01ad9 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.270427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 308/600] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:52.270427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9738b4c-fdf6-4f18-84d5-586e87f01ad9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:52.281314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:52.352743 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c48f4-b979-4402-a9a8-b90610bd1305 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:52.376616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9738b4c-fdf6-4f18-84d5-586e87f01ad9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:52.387460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c0e6d-31e7-407b-a88b-f17fee8bcce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:52.388090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c0e6d-31e7-407b-a88b-f17fee8bcce8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.388371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c0e6d-31e7-407b-a88b-f17fee8bcce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.388644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c0e6d-31e7-407b-a88b-f17fee8bcce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.399565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c0e6d-31e7-407b-a88b-f17fee8bcce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:52.401045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c0e6d-31e7-407b-a88b-f17fee8bcce8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:52.411506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036b625-8f1a-4117-9f56-668cb7fed059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:52.412239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036b625-8f1a-4117-9f56-668cb7fed059 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.412524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036b625-8f1a-4117-9f56-668cb7fed059 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.412797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036b625-8f1a-4117-9f56-668cb7fed059 None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/e32b5bf1a2524f1cbc432f38195e4c08/roles/691d078c15384159b1eec2b2ceaad9ed` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-419c0e6d-31e7-407b-a88b-f17fee8bcce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:52.414987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:52.416461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:52.417369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:52.417369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:52.417369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:52.417369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 293/601] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:52.417369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c0e6d-31e7-407b-a88b-f17fee8bcce8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:52.434319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036b625-8f1a-4117-9f56-668cb7fed059 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=e32b5bf1a2524f1cbc432f38195e4c08, role_id=691d078c15384159b1eec2b2ceaad9ed)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:52.435807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036b625-8f1a-4117-9f56-668cb7fed059 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:52.481009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:52.515915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 309/602] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:46:52 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/e32b5bf1a2524f1cbc432f38195e4c08/roles/691d078c15384159b1eec2b2ceaad9ed => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:52.515915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84776c05-1b07-410b-9c45-adcccbc02e0c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.515915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84776c05-1b07-410b-9c45-adcccbc02e0c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.515915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84776c05-1b07-410b-9c45-adcccbc02e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.532108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c0e6d-31e7-407b-a88b-f17fee8bcce8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:52.532744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-84776c05-1b07-410b-9c45-adcccbc02e0c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:52.535166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-84776c05-1b07-410b-9c45-adcccbc02e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:52.582863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b602c-6039-4642-a219-7f6e8c3c958f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:52.583492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b602c-6039-4642-a219-7f6e8c3c958f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.583767 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b602c-6039-4642-a219-7f6e8c3c958f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.584037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b602c-6039-4642-a219-7f6e8c3c958f None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.585053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b602c-6039-4642-a219-7f6e8c3c958f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:52.585381 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b602c-6039-4642-a219-7f6e8c3c958f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:52.587981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b602c-6039-4642-a219-7f6e8c3c958f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:52.605888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 294/603] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:52.606783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b602c-6039-4642-a219-7f6e8c3c958f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:52.621761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:52.639108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 310/604] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:52.640331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84776c05-1b07-410b-9c45-adcccbc02e0c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:52.761419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84776c05-1b07-410b-9c45-adcccbc02e0c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:52.769602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b602c-6039-4642-a219-7f6e8c3c958f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:52.797918 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c69963-7509-4c4a-81b8-af3fdbaecada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:52.799300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c69963-7509-4c4a-81b8-af3fdbaecada None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.799673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c69963-7509-4c4a-81b8-af3fdbaecada None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.799780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:52.800120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c69963-7509-4c4a-81b8-af3fdbaecada None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/fb514134abe54e5ca8125e572a2ff928/roles/691d078c15384159b1eec2b2ceaad9ed` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.810926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e7ddb-82ed-4016-b423-38019f4fa1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:52.811790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e7ddb-82ed-4016-b423-38019f4fa1bc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.812333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e7ddb-82ed-4016-b423-38019f4fa1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.813009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e7ddb-82ed-4016-b423-38019f4fa1bc None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.813942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e7ddb-82ed-4016-b423-38019f4fa1bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:52.814398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e7ddb-82ed-4016-b423-38019f4fa1bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:52.816131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e7ddb-82ed-4016-b423-38019f4fa1bc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:52.817965 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c69963-7509-4c4a-81b8-af3fdbaecada None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=fb514134abe54e5ca8125e572a2ff928, role_id=691d078c15384159b1eec2b2ceaad9ed)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:52.819528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c69963-7509-4c4a-81b8-af3fdbaecada None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:52.834152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 295/605] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:52.837174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e7ddb-82ed-4016-b423-38019f4fa1bc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:52.923779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 311/606] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:46:52 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/fb514134abe54e5ca8125e572a2ff928/roles/691d078c15384159b1eec2b2ceaad9ed => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:52.943808 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e7ddb-82ed-4016-b423-38019f4fa1bc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:52.994512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df490c-ebe5-42c3-9e53-dbedeee10f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:52.995196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df490c-ebe5-42c3-9e53-dbedeee10f5d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:52.995525 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df490c-ebe5-42c3-9e53-dbedeee10f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:52.995815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df490c-ebe5-42c3-9e53-dbedeee10f5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:52.995994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c69963-7509-4c4a-81b8-af3fdbaecada None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:53.006205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df490c-ebe5-42c3-9e53-dbedeee10f5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:53.007803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df490c-ebe5-42c3-9e53-dbedeee10f5d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-08df490c-ebe5-42c3-9e53-dbedeee10f5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:53.024761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:53.026445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:53.027562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:53.027562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:53.027562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:53.027562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:53.027562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:53.027562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:53.027562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:53.027562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:53.030769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 296/607] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:53.033770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df490c-ebe5-42c3-9e53-dbedeee10f5d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:53.062009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:53.125005 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:53.146389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c69963-7509-4c4a-81b8-af3fdbaecada None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:53.210663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79757f-297e-4200-b1f3-d2d758ada7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:53.211303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79757f-297e-4200-b1f3-d2d758ada7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:53.211685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79757f-297e-4200-b1f3-d2d758ada7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:53.211959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79757f-297e-4200-b1f3-d2d758ada7b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:53.219613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df490c-ebe5-42c3-9e53-dbedeee10f5d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:53.226161 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79757f-297e-4200-b1f3-d2d758ada7b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-de79757f-297e-4200-b1f3-d2d758ada7b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:53.231284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:53.232917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:53.233837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 312/608] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:53.238654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79757f-297e-4200-b1f3-d2d758ada7b0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:53.258133 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81e37e-494b-400b-a680-d9b6b98ea3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:53.258777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81e37e-494b-400b-a680-d9b6b98ea3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:53.259057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81e37e-494b-400b-a680-d9b6b98ea3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:53.259332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81e37e-494b-400b-a680-d9b6b98ea3c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:53.260572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd81e37e-494b-400b-a680-d9b6b98ea3c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:53.260874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd81e37e-494b-400b-a680-d9b6b98ea3c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:53.262320 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd81e37e-494b-400b-a680-d9b6b98ea3c4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:53.275807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 297/609] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:53.282475 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81e37e-494b-400b-a680-d9b6b98ea3c4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:53.357738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79757f-297e-4200-b1f3-d2d758ada7b0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:53.387182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:53.401371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4497bb-7903-42db-8af5-a7e0b80b2901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:53.406527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4497bb-7903-42db-8af5-a7e0b80b2901 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:53.406527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4497bb-7903-42db-8af5-a7e0b80b2901 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:53.406527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4497bb-7903-42db-8af5-a7e0b80b2901 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:53.406527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4497bb-7903-42db-8af5-a7e0b80b2901 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:53.406527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4497bb-7903-42db-8af5-a7e0b80b2901 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:53.408776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4497bb-7903-42db-8af5-a7e0b80b2901 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:53.425908 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 313/610] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:46:53.432197 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4497bb-7903-42db-8af5-a7e0b80b2901 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:53.495908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81e37e-494b-400b-a680-d9b6b98ea3c4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:53.498619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:53.565270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72e7bc-ada7-46a6-b6e8-3d4061cb2fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:53.566385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72e7bc-ada7-46a6-b6e8-3d4061cb2fe5 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:53.568087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72e7bc-ada7-46a6-b6e8-3d4061cb2fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:53.568612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72e7bc-ada7-46a6-b6e8-3d4061cb2fe5 None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/a29c4e53ac9a438e9701a70f3de3f32e/roles/691d078c15384159b1eec2b2ceaad9ed` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:53.599183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72e7bc-ada7-46a6-b6e8-3d4061cb2fe5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=a29c4e53ac9a438e9701a70f3de3f32e, role_id=691d078c15384159b1eec2b2ceaad9ed)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:53.600876 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72e7bc-ada7-46a6-b6e8-3d4061cb2fe5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:53.616306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4497bb-7903-42db-8af5-a7e0b80b2901 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:53.650614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4257e05-4c14-48a9-b574-47296fd57e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:53.651277 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4257e05-4c14-48a9-b574-47296fd57e3c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:53.651555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4257e05-4c14-48a9-b574-47296fd57e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:53.651824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4257e05-4c14-48a9-b574-47296fd57e3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:53.663128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4257e05-4c14-48a9-b574-47296fd57e3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:53.664578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4257e05-4c14-48a9-b574-47296fd57e3c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:53.678633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 298/611] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:46:53 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/a29c4e53ac9a438e9701a70f3de3f32e/roles/691d078c15384159b1eec2b2ceaad9ed => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:53.680368 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffffc3-c2d9-41e3-aa1f-76d3b045f5ee None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-c4257e05-4c14-48a9-b574-47296fd57e3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:53.680478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:53.688692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:53.689586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:53.689586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:53.689586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:53.689757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffffc3-c2d9-41e3-aa1f-76d3b045f5ee None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:53.689757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffffc3-c2d9-41e3-aa1f-76d3b045f5ee None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:53.689757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 299/612] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:53.689991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 314/613] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:53.691758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffffc3-c2d9-41e3-aa1f-76d3b045f5ee None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:53.692292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314038e2-1d27-4711-a0d0-1f9d6ff72145 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:53.692605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314038e2-1d27-4711-a0d0-1f9d6ff72145 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:53.692851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314038e2-1d27-4711-a0d0-1f9d6ff72145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:53.716020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-314038e2-1d27-4711-a0d0-1f9d6ff72145 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:53.717322 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-314038e2-1d27-4711-a0d0-1f9d6ff72145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:53.732205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:53.808212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 315/614] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:53.871478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffffc3-c2d9-41e3-aa1f-76d3b045f5ee None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:53.922378 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec28f-d314-4d65-a89b-c0c781e99e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:53.922588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec28f-d314-4d65-a89b-c0c781e99e3a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:53.922861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec28f-d314-4d65-a89b-c0c781e99e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:53.923102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec28f-d314-4d65-a89b-c0c781e99e3a None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:53.923755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ec28f-d314-4d65-a89b-c0c781e99e3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:53.924046 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ec28f-d314-4d65-a89b-c0c781e99e3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:53.925476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ec28f-d314-4d65-a89b-c0c781e99e3a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:53.942607 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 300/615] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:53.947969 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314038e2-1d27-4711-a0d0-1f9d6ff72145 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:54.053750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314038e2-1d27-4711-a0d0-1f9d6ff72145 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:54.095266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafe372-3394-45a1-b1e5-29268e82c5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:54.096140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe372-3394-45a1-b1e5-29268e82c5fc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:54.096560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe372-3394-45a1-b1e5-29268e82c5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:54.096970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe372-3394-45a1-b1e5-29268e82c5fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:54.105226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbafe372-3394-45a1-b1e5-29268e82c5fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:54.106879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbafe372-3394-45a1-b1e5-29268e82c5fc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-fbafe372-3394-45a1-b1e5-29268e82c5fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:54.118715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:54.120417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:54.121378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:54.121378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:54.121378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:54.121378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:54.121378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:54.121378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:54.121378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:54.121378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:54.121378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:46:54.121378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:54.123464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 316/616] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:46:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:54.131993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec28f-d314-4d65-a89b-c0c781e99e3a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:54.152147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:54.230073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafe372-3394-45a1-b1e5-29268e82c5fc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:54.234759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:54.271291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec28f-d314-4d65-a89b-c0c781e99e3a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:54.310146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688da55-8647-4b22-83ec-791bf8803165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:54.311181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688da55-8647-4b22-83ec-791bf8803165 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:54.311678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688da55-8647-4b22-83ec-791bf8803165 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:54.312093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688da55-8647-4b22-83ec-791bf8803165 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:54.313213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688da55-8647-4b22-83ec-791bf8803165 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:54.313658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688da55-8647-4b22-83ec-791bf8803165 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:54.315390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688da55-8647-4b22-83ec-791bf8803165 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:54.334208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 301/617] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Sep 16 07:46:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:54.340744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688da55-8647-4b22-83ec-791bf8803165 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:54.447742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:54.475504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafe372-3394-45a1-b1e5-29268e82c5fc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:54.489484 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688da55-8647-4b22-83ec-791bf8803165 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:54.532194 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b50cc8-8dae-4a4c-aee1-bb7c7f5c1423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:54.532995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b50cc8-8dae-4a4c-aee1-bb7c7f5c1423 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:54.533516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b50cc8-8dae-4a4c-aee1-bb7c7f5c1423 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:54.533791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b50cc8-8dae-4a4c-aee1-bb7c7f5c1423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:54.538009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d7605-afb9-49f9-9ce0-a981aabe5957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:54.538946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d7605-afb9-49f9-9ce0-a981aabe5957 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:54.539449 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d7605-afb9-49f9-9ce0-a981aabe5957 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:54.539922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d7605-afb9-49f9-9ce0-a981aabe5957 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:54.547479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b50cc8-8dae-4a4c-aee1-bb7c7f5c1423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:54.549255 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b50cc8-8dae-4a4c-aee1-bb7c7f5c1423 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:54.557730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d7605-afb9-49f9-9ce0-a981aabe5957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-667d7605-afb9-49f9-9ce0-a981aabe5957 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:54.568105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:54.569206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:54.570130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:54.571699 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 317/618] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:46:54.578226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d7605-afb9-49f9-9ce0-a981aabe5957 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-31b50cc8-8dae-4a4c-aee1-bb7c7f5c1423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:54.583949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:54.584878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:54.585790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:54.585790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:54.585790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:54.585790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 302/619] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:54.592286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b50cc8-8dae-4a4c-aee1-bb7c7f5c1423 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:54.673041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:54.753130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d7605-afb9-49f9-9ce0-a981aabe5957 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:54.800606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b50cc8-8dae-4a4c-aee1-bb7c7f5c1423 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:54.819658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b691f10-24a9-4296-a380-129e44dda3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:54.820355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b691f10-24a9-4296-a380-129e44dda3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:54.820549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b691f10-24a9-4296-a380-129e44dda3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:54.820824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b691f10-24a9-4296-a380-129e44dda3b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:54.821493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b691f10-24a9-4296-a380-129e44dda3b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:54.822382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b691f10-24a9-4296-a380-129e44dda3b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:54.823723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b691f10-24a9-4296-a380-129e44dda3b5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:54.842668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 318/620] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:54.847833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b691f10-24a9-4296-a380-129e44dda3b5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:54.850605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:54.870081 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a079c6-8599-4380-886f-c945c4b82b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:54.870712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a079c6-8599-4380-886f-c945c4b82b19 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:54.870995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a079c6-8599-4380-886f-c945c4b82b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:54.871266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a079c6-8599-4380-886f-c945c4b82b19 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:54.871917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a079c6-8599-4380-886f-c945c4b82b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:54.872224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a079c6-8599-4380-886f-c945c4b82b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:54.873643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a079c6-8599-4380-886f-c945c4b82b19 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:54.896550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 303/621] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:54.904470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a079c6-8599-4380-886f-c945c4b82b19 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:55.048174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b691f10-24a9-4296-a380-129e44dda3b5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:55.085046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:55.091869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a079c6-8599-4380-886f-c945c4b82b19 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:55.115392 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81feb8-7e83-4b6f-80be-e63071d1c91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:55.116422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81feb8-7e83-4b6f-80be-e63071d1c91c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.116915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81feb8-7e83-4b6f-80be-e63071d1c91c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.117396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81feb8-7e83-4b6f-80be-e63071d1c91c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.128412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4aa2d0-bdf2-4bf6-9f08-8a746bd0c707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:55.129059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4aa2d0-bdf2-4bf6-9f08-8a746bd0c707 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.129343 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4aa2d0-bdf2-4bf6-9f08-8a746bd0c707 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.129623 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4aa2d0-bdf2-4bf6-9f08-8a746bd0c707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.133655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81feb8-7e83-4b6f-80be-e63071d1c91c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:55.135938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81feb8-7e83-4b6f-80be-e63071d1c91c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:55.137247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4aa2d0-bdf2-4bf6-9f08-8a746bd0c707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:55.138745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4aa2d0-bdf2-4bf6-9f08-8a746bd0c707 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9c4aa2d0-bdf2-4bf6-9f08-8a746bd0c707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:55.151115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:55.152056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:55.152974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:55.152974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:55.152974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:55.152974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:55.152974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:55.152974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:55.152974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:55.152974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:55.152974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 304/622] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:55.154988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a8e80-1a89-43cd-9a6d-4b6d0c94c69f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.154988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a8e80-1a89-43cd-9a6d-4b6d0c94c69f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.154988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a8e80-1a89-43cd-9a6d-4b6d0c94c69f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-bb81feb8-7e83-4b6f-80be-e63071d1c91c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:55.155746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:55.156654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:55.157547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:55.157547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:55.157547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:55.157715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 305/623] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:55.160763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 319/624] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:55.161765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81feb8-7e83-4b6f-80be-e63071d1c91c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:55.162126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f7038-5956-46df-bcca-ec604f5030be None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.162416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f7038-5956-46df-bcca-ec604f5030be None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.162707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f7038-5956-46df-bcca-ec604f5030be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.182953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c70f7038-5956-46df-bcca-ec604f5030be None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:55.184205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c70f7038-5956-46df-bcca-ec604f5030be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:55.224511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:55.263908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 306/625] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:55.263908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f7038-5956-46df-bcca-ec604f5030be None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:55.303337 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81feb8-7e83-4b6f-80be-e63071d1c91c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:55.382042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12063f-8ead-4c2d-91eb-5a70367432ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:55.382660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12063f-8ead-4c2d-91eb-5a70367432ac None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.382956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12063f-8ead-4c2d-91eb-5a70367432ac None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.383235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12063f-8ead-4c2d-91eb-5a70367432ac None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.383941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12063f-8ead-4c2d-91eb-5a70367432ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:55.384245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12063f-8ead-4c2d-91eb-5a70367432ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:55.386256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12063f-8ead-4c2d-91eb-5a70367432ac None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:55.405935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f7038-5956-46df-bcca-ec604f5030be None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:55.413583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 320/626] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:55.419731 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12063f-8ead-4c2d-91eb-5a70367432ac None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:55.472836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4db56e-c34a-4322-bbd1-76cc952c8986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:55.473452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db56e-c34a-4322-bbd1-76cc952c8986 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.473740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db56e-c34a-4322-bbd1-76cc952c8986 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.474014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4db56e-c34a-4322-bbd1-76cc952c8986 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.474667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4db56e-c34a-4322-bbd1-76cc952c8986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:55.474995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4db56e-c34a-4322-bbd1-76cc952c8986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:55.476368 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4db56e-c34a-4322-bbd1-76cc952c8986 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:55.494171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:55.503106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 307/627] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:55.509219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4db56e-c34a-4322-bbd1-76cc952c8986 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:55.547626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:55.568602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12063f-8ead-4c2d-91eb-5a70367432ac None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:55.605708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584081c-95bd-4bd6-8020-fa634c87a655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:55.606373 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584081c-95bd-4bd6-8020-fa634c87a655 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.606656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584081c-95bd-4bd6-8020-fa634c87a655 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.606958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584081c-95bd-4bd6-8020-fa634c87a655 None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/a1ae39fdd6ff4841843c9826dd88229a/roles/691d078c15384159b1eec2b2ceaad9ed` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.620097 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4db56e-c34a-4322-bbd1-76cc952c8986 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:55.620936 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584081c-95bd-4bd6-8020-fa634c87a655 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=a1ae39fdd6ff4841843c9826dd88229a, role_id=691d078c15384159b1eec2b2ceaad9ed)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:55.622388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584081c-95bd-4bd6-8020-fa634c87a655 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:55.654419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29203123-7d62-4ebf-b53a-862a3a0fe448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:55.655044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29203123-7d62-4ebf-b53a-862a3a0fe448 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.655322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29203123-7d62-4ebf-b53a-862a3a0fe448 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.655597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29203123-7d62-4ebf-b53a-862a3a0fe448 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.662880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29203123-7d62-4ebf-b53a-862a3a0fe448 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:55.665891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29203123-7d62-4ebf-b53a-862a3a0fe448 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-29203123-7d62-4ebf-b53a-862a3a0fe448 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:55.675147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:55.676149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:55.677170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:55.677170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:55.677170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:55.677170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:55.677170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:55.677170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:55.677170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:55.677170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:55.677170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 16 07:46:55.677170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:55.677170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 308/628] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:55.677784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5b4fa-9406-4891-b902-0c9bc4b44a91 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.677784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5b4fa-9406-4891-b902-0c9bc4b44a91 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.678252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5b4fa-9406-4891-b902-0c9bc4b44a91 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.681019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 309/629] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:55.681019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5b4fa-9406-4891-b902-0c9bc4b44a91 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:55.692623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 321/630] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:46:55 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/a1ae39fdd6ff4841843c9826dd88229a/roles/691d078c15384159b1eec2b2ceaad9ed => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:46:55.704558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584081c-95bd-4bd6-8020-fa634c87a655 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:55.807318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:55.823485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5b4fa-9406-4891-b902-0c9bc4b44a91 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:55.827581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:55.841244 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584081c-95bd-4bd6-8020-fa634c87a655 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:55.871002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe16ea3-501b-452c-aabf-e04d51e84b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:55.871672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe16ea3-501b-452c-aabf-e04d51e84b02 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.871952 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe16ea3-501b-452c-aabf-e04d51e84b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.872231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe16ea3-501b-452c-aabf-e04d51e84b02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.886638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe16ea3-501b-452c-aabf-e04d51e84b02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:55.887536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a87d53-a14e-4748-ba00-64dd9efbf40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:55.889083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a87d53-a14e-4748-ba00-64dd9efbf40e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.889083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a87d53-a14e-4748-ba00-64dd9efbf40e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.889083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a87d53-a14e-4748-ba00-64dd9efbf40e None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.889916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a87d53-a14e-4748-ba00-64dd9efbf40e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:55.890228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a87d53-a14e-4748-ba00-64dd9efbf40e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-abe16ea3-501b-452c-aabf-e04d51e84b02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:55.892030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:55.892965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:55.894569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:55.895263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a87d53-a14e-4748-ba00-64dd9efbf40e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:55.895337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 310/631] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:55.895337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff00d4-b3da-48f3-b21f-8638718cd8d9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:55.895337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff00d4-b3da-48f3-b21f-8638718cd8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:55.895337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff00d4-b3da-48f3-b21f-8638718cd8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:55.917587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 322/632] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Sep 16 07:46:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:55.918142 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a87d53-a14e-4748-ba00-64dd9efbf40e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:55.919626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-04ff00d4-b3da-48f3-b21f-8638718cd8d9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:55.921262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-04ff00d4-b3da-48f3-b21f-8638718cd8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:56.001894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 311/633] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:56.002903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff00d4-b3da-48f3-b21f-8638718cd8d9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:56.058891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a87d53-a14e-4748-ba00-64dd9efbf40e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:56.119843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:56.139542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff00d4-b3da-48f3-b21f-8638718cd8d9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:56.157660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe0a38-0d62-4a97-a251-d23040b1dfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:56.158585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe0a38-0d62-4a97-a251-d23040b1dfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:56.159056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe0a38-0d62-4a97-a251-d23040b1dfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:56.159497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe0a38-0d62-4a97-a251-d23040b1dfb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:56.160332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe0a38-0d62-4a97-a251-d23040b1dfb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:56.161180 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe0a38-0d62-4a97-a251-d23040b1dfb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:56.163626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe0a38-0d62-4a97-a251-d23040b1dfb3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:56.184521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 323/634] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:56.189309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe0a38-0d62-4a97-a251-d23040b1dfb3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:56.207146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:56.208689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60348d-f6bd-461f-8018-e1fdef33f482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:56.209547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60348d-f6bd-461f-8018-e1fdef33f482 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:56.210040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60348d-f6bd-461f-8018-e1fdef33f482 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:56.210520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60348d-f6bd-461f-8018-e1fdef33f482 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:56.332996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60348d-f6bd-461f-8018-e1fdef33f482 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:56.335682 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60348d-f6bd-461f-8018-e1fdef33f482 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:56.366876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe0a38-0d62-4a97-a251-d23040b1dfb3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:56.403305 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60afc78-058c-482b-be7a-ed84c3381032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:56.403883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60afc78-058c-482b-be7a-ed84c3381032 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:56.404148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60afc78-058c-482b-be7a-ed84c3381032 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:56.404424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60afc78-058c-482b-be7a-ed84c3381032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:56.415019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60afc78-058c-482b-be7a-ed84c3381032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:56.416486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60afc78-058c-482b-be7a-ed84c3381032 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:56.425612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b60afc78-058c-482b-be7a-ed84c3381032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:56.431891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:56.437820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:56.438979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:56.438979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:56.438979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:56.438979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 324/635] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:56.438979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60afc78-058c-482b-be7a-ed84c3381032 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:56.452612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 312/636] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:56.453964 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d61ad-bce2-4509-91d0-7994a4328097 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:56.454323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d61ad-bce2-4509-91d0-7994a4328097 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:56.454692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d61ad-bce2-4509-91d0-7994a4328097 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:56.456167 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 313/637] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:56.456748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d61ad-bce2-4509-91d0-7994a4328097 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:56.504621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:56.546748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60afc78-058c-482b-be7a-ed84c3381032 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:56.565578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d61ad-bce2-4509-91d0-7994a4328097 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:56.584216 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c95877-fc87-48fc-b712-f749d2758d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:56.584855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95877-fc87-48fc-b712-f749d2758d52 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:56.585136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95877-fc87-48fc-b712-f749d2758d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:56.585419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95877-fc87-48fc-b712-f749d2758d52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:56.596059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c95877-fc87-48fc-b712-f749d2758d52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:56.599552 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ab20b-1a82-48a1-ae8c-53cf6c5a9832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:56.600155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ab20b-1a82-48a1-ae8c-53cf6c5a9832 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:56.600435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ab20b-1a82-48a1-ae8c-53cf6c5a9832 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:56.600706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ab20b-1a82-48a1-ae8c-53cf6c5a9832 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b0c95877-fc87-48fc-b712-f749d2758d52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:56.601303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:56.602228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:56.603122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:56.603847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ab20b-1a82-48a1-ae8c-53cf6c5a9832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:56.603847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ab20b-1a82-48a1-ae8c-53cf6c5a9832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:56.603972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 325/638] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:56.604037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ab20b-1a82-48a1-ae8c-53cf6c5a9832 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:56.604103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c95877-fc87-48fc-b712-f749d2758d52 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:56.620142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 314/639] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:56.621497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624e884-e086-4805-9e04-00af64289ac8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:56.621784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624e884-e086-4805-9e04-00af64289ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:56.622056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624e884-e086-4805-9e04-00af64289ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:56.640016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9624e884-e086-4805-9e04-00af64289ac8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:56.641364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9624e884-e086-4805-9e04-00af64289ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:56.678189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:56.712155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 315/640] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:56.713075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624e884-e086-4805-9e04-00af64289ac8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:56.728982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c95877-fc87-48fc-b712-f749d2758d52 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:56.758537 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:56.775582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2540c3-d192-4f29-b6a7-1988bb758fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:56.776080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2540c3-d192-4f29-b6a7-1988bb758fac None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:56.776951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2540c3-d192-4f29-b6a7-1988bb758fac None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:56.777322 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2540c3-d192-4f29-b6a7-1988bb758fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:56.789065 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2540c3-d192-4f29-b6a7-1988bb758fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:56.792145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2540c3-d192-4f29-b6a7-1988bb758fac None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-cb2540c3-d192-4f29-b6a7-1988bb758fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:56.812689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:56.813660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:56.814567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:56.814567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:56.814567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:56.814567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 326/641] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:56.814853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2540c3-d192-4f29-b6a7-1988bb758fac None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:56.836009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624e884-e086-4805-9e04-00af64289ac8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:56.884035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170a184-5e24-409e-ba70-d6567f052411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:56.885142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170a184-5e24-409e-ba70-d6567f052411 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:56.885629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170a184-5e24-409e-ba70-d6567f052411 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:56.886080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170a184-5e24-409e-ba70-d6567f052411 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:56.886951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170a184-5e24-409e-ba70-d6567f052411 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:56.887435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170a184-5e24-409e-ba70-d6567f052411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:56.888976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170a184-5e24-409e-ba70-d6567f052411 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:56.897393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 316/642] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:56.898441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170a184-5e24-409e-ba70-d6567f052411 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:56.942256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:56.945162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2540c3-d192-4f29-b6a7-1988bb758fac None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:56.981716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644399b4-278e-4f53-b5b1-2a256ef8dbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:56.982734 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644399b4-278e-4f53-b5b1-2a256ef8dbdd None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:56.983259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644399b4-278e-4f53-b5b1-2a256ef8dbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:56.983772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644399b4-278e-4f53-b5b1-2a256ef8dbdd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:56.993120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644399b4-278e-4f53-b5b1-2a256ef8dbdd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:56.995466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644399b4-278e-4f53-b5b1-2a256ef8dbdd None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-644399b4-278e-4f53-b5b1-2a256ef8dbdd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:57.007210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:57.008229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:57.009186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:57.009186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:57.009186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:57.009186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:57.009186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:57.009186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:57.009186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:57.009186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:57.009186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:46:57.009186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:57.011755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 327/643] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Sep 16 07:46:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:57.012951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644399b4-278e-4f53-b5b1-2a256ef8dbdd None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:57.023536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170a184-5e24-409e-ba70-d6567f052411 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:57.045656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:57.061027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2108c-b5aa-4d78-b91f-14704f61101f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:57.061775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2108c-b5aa-4d78-b91f-14704f61101f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:57.062151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2108c-b5aa-4d78-b91f-14704f61101f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:57.062797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2108c-b5aa-4d78-b91f-14704f61101f None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:57.063443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2108c-b5aa-4d78-b91f-14704f61101f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:57.063745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2108c-b5aa-4d78-b91f-14704f61101f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:57.065128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2108c-b5aa-4d78-b91f-14704f61101f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:57.077123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 317/644] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:57.078041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2108c-b5aa-4d78-b91f-14704f61101f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:57.123525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644399b4-278e-4f53-b5b1-2a256ef8dbdd None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:57.159777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d724dfe-f98f-47b8-b9af-02f986ccd050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:57.160765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724dfe-f98f-47b8-b9af-02f986ccd050 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:57.161048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724dfe-f98f-47b8-b9af-02f986ccd050 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:57.161629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724dfe-f98f-47b8-b9af-02f986ccd050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:57.173183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d724dfe-f98f-47b8-b9af-02f986ccd050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:57.174612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d724dfe-f98f-47b8-b9af-02f986ccd050 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:57.177649 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:57.184357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2108c-b5aa-4d78-b91f-14704f61101f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-0d724dfe-f98f-47b8-b9af-02f986ccd050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:57.188859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:57.190619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:57.191938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:57.191938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:57.191938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:57.191938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 328/645] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:57.191938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d724dfe-f98f-47b8-b9af-02f986ccd050 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:57.219212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef42635-7ddc-4a33-aa74-9ae6bec2c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:57.219753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef42635-7ddc-4a33-aa74-9ae6bec2c07d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:57.220025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef42635-7ddc-4a33-aa74-9ae6bec2c07d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:57.220295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef42635-7ddc-4a33-aa74-9ae6bec2c07d None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:57.221011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef42635-7ddc-4a33-aa74-9ae6bec2c07d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:57.221340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef42635-7ddc-4a33-aa74-9ae6bec2c07d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:57.222789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef42635-7ddc-4a33-aa74-9ae6bec2c07d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:57.241577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 318/646] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Sep 16 07:46:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:57.242368 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef42635-7ddc-4a33-aa74-9ae6bec2c07d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:57.287593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:57.304541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d724dfe-f98f-47b8-b9af-02f986ccd050 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:57.340175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d87f2-ac24-476a-95ee-a18bc236d807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:57.341025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d87f2-ac24-476a-95ee-a18bc236d807 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:57.341309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d87f2-ac24-476a-95ee-a18bc236d807 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:57.341587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d87f2-ac24-476a-95ee-a18bc236d807 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:57.351472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef42635-7ddc-4a33-aa74-9ae6bec2c07d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:57.351915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d87f2-ac24-476a-95ee-a18bc236d807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-a35d87f2-ac24-476a-95ee-a18bc236d807 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:57.357727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:57.362860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 329/647] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:57.363777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d87f2-ac24-476a-95ee-a18bc236d807 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:57.395074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76f5d5-552b-4fe6-83f4-66bcce80d2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:57.395965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76f5d5-552b-4fe6-83f4-66bcce80d2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:57.396243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76f5d5-552b-4fe6-83f4-66bcce80d2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:57.396513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76f5d5-552b-4fe6-83f4-66bcce80d2f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:57.403753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76f5d5-552b-4fe6-83f4-66bcce80d2f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:57.405189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76f5d5-552b-4fe6-83f4-66bcce80d2f2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9e76f5d5-552b-4fe6-83f4-66bcce80d2f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:57.417995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:57.418935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:57.419831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:57.419831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:57.419831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:57.419831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:57.419831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:57.419831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:57.419831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:57.419831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:57.419831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 319/648] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:57.420317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76f5d5-552b-4fe6-83f4-66bcce80d2f2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:57.435109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:57.474734 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d87f2-ac24-476a-95ee-a18bc236d807 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:57.511842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98962f6-6926-42b4-9208-4c776aa3a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:57.512472 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98962f6-6926-42b4-9208-4c776aa3a743 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:57.512738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98962f6-6926-42b4-9208-4c776aa3a743 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:57.513013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98962f6-6926-42b4-9208-4c776aa3a743 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:57.513661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98962f6-6926-42b4-9208-4c776aa3a743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:57.513967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98962f6-6926-42b4-9208-4c776aa3a743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:57.515356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98962f6-6926-42b4-9208-4c776aa3a743 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:57.522004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76f5d5-552b-4fe6-83f4-66bcce80d2f2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:57.532863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 330/649] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:57.532863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98962f6-6926-42b4-9208-4c776aa3a743 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:57.534848 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:57.560728 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123c2db-dec7-4250-a858-d708cd2c1dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:57.561338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123c2db-dec7-4250-a858-d708cd2c1dda None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:57.561612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123c2db-dec7-4250-a858-d708cd2c1dda None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:57.561882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123c2db-dec7-4250-a858-d708cd2c1dda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:57.629537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123c2db-dec7-4250-a858-d708cd2c1dda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:57.631018 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123c2db-dec7-4250-a858-d708cd2c1dda None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:57.633541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98962f6-6926-42b4-9208-4c776aa3a743 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:57.645138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:57.671647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf6bd6-8bcc-4862-b523-a133fa51adb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:57.672292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf6bd6-8bcc-4862-b523-a133fa51adb8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:57.672570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf6bd6-8bcc-4862-b523-a133fa51adb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:57.672845 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf6bd6-8bcc-4862-b523-a133fa51adb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:57.673499 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf6bd6-8bcc-4862-b523-a133fa51adb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:57.673828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf6bd6-8bcc-4862-b523-a133fa51adb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:57.675187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf6bd6-8bcc-4862-b523-a133fa51adb8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:57.684025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 331/650] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:57.684800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf6bd6-8bcc-4862-b523-a133fa51adb8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:57.748660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 320/651] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:57.748660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123c2db-dec7-4250-a858-d708cd2c1dda None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:57.768026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:57.811516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf6bd6-8bcc-4862-b523-a133fa51adb8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:57.848431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123c2db-dec7-4250-a858-d708cd2c1dda None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:57.848619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34303b9-64b7-4b6a-958e-a91827b89b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:57.849184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34303b9-64b7-4b6a-958e-a91827b89b4d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:57.849468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34303b9-64b7-4b6a-958e-a91827b89b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:57.849859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34303b9-64b7-4b6a-958e-a91827b89b4d None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:57.850669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34303b9-64b7-4b6a-958e-a91827b89b4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:57.851023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34303b9-64b7-4b6a-958e-a91827b89b4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:57.852507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34303b9-64b7-4b6a-958e-a91827b89b4d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:57.862270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:57.867329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 332/652] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:57.868401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34303b9-64b7-4b6a-958e-a91827b89b4d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:57.892825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47841a-8882-4c06-8c89-394186380e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:57.893574 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47841a-8882-4c06-8c89-394186380e84 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:57.893932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47841a-8882-4c06-8c89-394186380e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:57.894319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47841a-8882-4c06-8c89-394186380e84 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:57.902940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47841a-8882-4c06-8c89-394186380e84 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:57.904410 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47841a-8882-4c06-8c89-394186380e84 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8c47841a-8882-4c06-8c89-394186380e84 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:57.918701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:57.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:57.921102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:57.921102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:57.921102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:57.921102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:57.921102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:57.921102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:57.921102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:57.921102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:57.921102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 07:46:57.921102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:57.922987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 321/653] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:46:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:57.923811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47841a-8882-4c06-8c89-394186380e84 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:57.986656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34303b9-64b7-4b6a-958e-a91827b89b4d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:58.012042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:58.020578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c006f5-bfa2-466f-babd-a4c3c2087df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:58.025831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c006f5-bfa2-466f-babd-a4c3c2087df3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.025831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c006f5-bfa2-466f-babd-a4c3c2087df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.025831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c006f5-bfa2-466f-babd-a4c3c2087df3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.028507 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47841a-8882-4c06-8c89-394186380e84 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:58.031956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c006f5-bfa2-466f-babd-a4c3c2087df3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:58.033375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c006f5-bfa2-466f-babd-a4c3c2087df3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-46c006f5-bfa2-466f-babd-a4c3c2087df3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:58.048340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:58.049454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:58.050649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:58.050649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:58.050649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:58.051099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 333/654] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:58.051966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c006f5-bfa2-466f-babd-a4c3c2087df3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:58.063942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b03850-b0aa-4af8-99e5-5a6ccef6849d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:58.064579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b03850-b0aa-4af8-99e5-5a6ccef6849d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.064854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b03850-b0aa-4af8-99e5-5a6ccef6849d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.065127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b03850-b0aa-4af8-99e5-5a6ccef6849d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.075416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b03850-b0aa-4af8-99e5-5a6ccef6849d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:58.076848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b03850-b0aa-4af8-99e5-5a6ccef6849d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a7b03850-b0aa-4af8-99e5-5a6ccef6849d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:58.090947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:58.092071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:58.093204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:58.093204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:58.093204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:58.093204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 322/655] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:58.093204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b03850-b0aa-4af8-99e5-5a6ccef6849d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:58.118066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:58.153609 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c006f5-bfa2-466f-babd-a4c3c2087df3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:58.186636 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41729524-4b1f-4802-9f0a-eb21c338427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:58.187259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729524-4b1f-4802-9f0a-eb21c338427b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.187813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729524-4b1f-4802-9f0a-eb21c338427b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.188066 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729524-4b1f-4802-9f0a-eb21c338427b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.188775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41729524-4b1f-4802-9f0a-eb21c338427b None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=c214b2c97a7b41698b433eac8379442a) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:58.189080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41729524-4b1f-4802-9f0a-eb21c338427b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:58.189759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b03850-b0aa-4af8-99e5-5a6ccef6849d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:58.190499 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41729524-4b1f-4802-9f0a-eb21c338427b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:58.196048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 334/656] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:58 2022] GET /identity/v3/role_assignments?user.id=c214b2c97a7b41698b433eac8379442a&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:58.197215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aed2a-4af0-4951-b3b5-5b96cddea247 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.197495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aed2a-4af0-4951-b3b5-5b96cddea247 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.197766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aed2a-4af0-4951-b3b5-5b96cddea247 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.198965 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 335/657] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:46:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:46:58.199654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406aed2a-4af0-4951-b3b5-5b96cddea247 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:58.242548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c0115-6590-4d44-a5dc-be0e7ef12d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:58.243470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c0115-6590-4d44-a5dc-be0e7ef12d8c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.244244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c0115-6590-4d44-a5dc-be0e7ef12d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.244516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c0115-6590-4d44-a5dc-be0e7ef12d8c None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.245480 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c0115-6590-4d44-a5dc-be0e7ef12d8c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:58.245608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c0115-6590-4d44-a5dc-be0e7ef12d8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:58.247066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c0115-6590-4d44-a5dc-be0e7ef12d8c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:58.255133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:58.270928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 323/658] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Sep 16 07:46:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:58.270928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c0115-6590-4d44-a5dc-be0e7ef12d8c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:58.321028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406aed2a-4af0-4951-b3b5-5b96cddea247 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:58.360294 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:58.360631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f0864-92ab-441b-9692-320d8ba8449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:58.361235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0864-92ab-441b-9692-320d8ba8449c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.361514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0864-92ab-441b-9692-320d8ba8449c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.361785 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f0864-92ab-441b-9692-320d8ba8449c None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.362548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f0864-92ab-441b-9692-320d8ba8449c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:58.362878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f0864-92ab-441b-9692-320d8ba8449c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:58.364278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f0864-92ab-441b-9692-320d8ba8449c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:58.384218 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 336/659] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:58.384436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c0115-6590-4d44-a5dc-be0e7ef12d8c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:58.384538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072cc4-a53d-48b0-8be2-26263e55c913 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.384538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072cc4-a53d-48b0-8be2-26263e55c913 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.384538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072cc4-a53d-48b0-8be2-26263e55c913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.402225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-dc072cc4-a53d-48b0-8be2-26263e55c913 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:46:58.403480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dc072cc4-a53d-48b0-8be2-26263e55c913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:46:58.419224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048834d-dbfa-406f-8c4a-344cd0f2dd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:58.419967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048834d-dbfa-406f-8c4a-344cd0f2dd0c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.420334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048834d-dbfa-406f-8c4a-344cd0f2dd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.420669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048834d-dbfa-406f-8c4a-344cd0f2dd0c None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.421314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048834d-dbfa-406f-8c4a-344cd0f2dd0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:58.421616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048834d-dbfa-406f-8c4a-344cd0f2dd0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:58.422994 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048834d-dbfa-406f-8c4a-344cd0f2dd0c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:58.435220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 324/660] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:58.436037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048834d-dbfa-406f-8c4a-344cd0f2dd0c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:58.481004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 337/661] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:46:58.481926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc072cc4-a53d-48b0-8be2-26263e55c913 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:58.519418 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:58.548232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048834d-dbfa-406f-8c4a-344cd0f2dd0c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:58.585926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cb2f4-e175-40ce-a776-40c3641ada8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:58.586916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cb2f4-e175-40ce-a776-40c3641ada8c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.587435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cb2f4-e175-40ce-a776-40c3641ada8c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.587908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cb2f4-e175-40ce-a776-40c3641ada8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.599036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc072cc4-a53d-48b0-8be2-26263e55c913 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:58.631088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:58.646643 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3c91c-1dac-410c-a3be-5e6e0b02d18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:58.647749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3c91c-1dac-410c-a3be-5e6e0b02d18c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.648124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3c91c-1dac-410c-a3be-5e6e0b02d18c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.648652 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3c91c-1dac-410c-a3be-5e6e0b02d18c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.659076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3c91c-1dac-410c-a3be-5e6e0b02d18c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:58.661467 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3c91c-1dac-410c-a3be-5e6e0b02d18c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:58.661961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cb2f4-e175-40ce-a776-40c3641ada8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:58.663436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cb2f4-e175-40ce-a776-40c3641ada8c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-f3c3c91c-1dac-410c-a3be-5e6e0b02d18c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:46:58.676588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:58.677739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:58.678958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:58.678958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:58.678958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:46:58.678958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:46:58.678958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:46:58.678958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:46:58.678958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:46:58.678958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:46:58.678958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 07:46:58.678958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:58.682868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 338/662] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:58.684844 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3c91c-1dac-410c-a3be-5e6e0b02d18c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:58.792056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:58.796721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 325/663] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:58.797562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cb2f4-e175-40ce-a776-40c3641ada8c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:58.806628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3c91c-1dac-410c-a3be-5e6e0b02d18c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:58.843871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:58.845792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5061f-4919-4b01-850e-b8c7d41f3f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:58.846543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5061f-4919-4b01-850e-b8c7d41f3f5f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.846933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5061f-4919-4b01-850e-b8c7d41f3f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.847313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5061f-4919-4b01-850e-b8c7d41f3f5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.855072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5061f-4919-4b01-850e-b8c7d41f3f5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:58.856592 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5061f-4919-4b01-850e-b8c7d41f3f5f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-76e5061f-4919-4b01-850e-b8c7d41f3f5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:58.869114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:58.870178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:58.871302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:58.871302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:58.871302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:58.871302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:58.871302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:58.871302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:58.871302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:58.871302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:58.873457 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 339/664] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:58.873868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5061f-4919-4b01-850e-b8c7d41f3f5f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:58.910310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cb2f4-e175-40ce-a776-40c3641ada8c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:58.949473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0de9ea-039b-4657-aedf-9e5680dabb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:58.950108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0de9ea-039b-4657-aedf-9e5680dabb81 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:58.950384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0de9ea-039b-4657-aedf-9e5680dabb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:58.950656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0de9ea-039b-4657-aedf-9e5680dabb81 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:58.951414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0de9ea-039b-4657-aedf-9e5680dabb81 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:58.951717 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0de9ea-039b-4657-aedf-9e5680dabb81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:58.953146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0de9ea-039b-4657-aedf-9e5680dabb81 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:58.972364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 326/665] 10.222.0.59 () {56 vars in 1139 bytes} [Fri Sep 16 07:46:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:58.973298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0de9ea-039b-4657-aedf-9e5680dabb81 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:58.984987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5061f-4919-4b01-850e-b8c7d41f3f5f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:59.021228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330aedd-9333-4225-b561-ffc4064d0cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:59.021875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330aedd-9333-4225-b561-ffc4064d0cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:59.022132 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330aedd-9333-4225-b561-ffc4064d0cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:59.022409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330aedd-9333-4225-b561-ffc4064d0cf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:59.034325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330aedd-9333-4225-b561-ffc4064d0cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-8330aedd-9333-4225-b561-ffc4064d0cf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:46:59.039587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:59.041097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:59.041985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 340/666] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:46:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:59.043383 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330aedd-9333-4225-b561-ffc4064d0cf0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:59.060502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:59.078523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:59.081554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0de9ea-039b-4657-aedf-9e5680dabb81 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:59.115217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edfc10-51cb-4a76-aa4f-0130bae1a6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:59.115861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edfc10-51cb-4a76-aa4f-0130bae1a6bf None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:59.116136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edfc10-51cb-4a76-aa4f-0130bae1a6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:59.116405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edfc10-51cb-4a76-aa4f-0130bae1a6bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:59.126879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edfc10-51cb-4a76-aa4f-0130bae1a6bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:59.128330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edfc10-51cb-4a76-aa4f-0130bae1a6bf None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-75edfc10-51cb-4a76-aa4f-0130bae1a6bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:59.143518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:59.144802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:59.145744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:59.145744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:59.145744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:59.145744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 327/667] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:59.145990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edfc10-51cb-4a76-aa4f-0130bae1a6bf None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:59.158706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330aedd-9333-4225-b561-ffc4064d0cf0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:59.197255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19716948-efe3-4eb1-bfca-0a3cdbf3bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:59.197893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19716948-efe3-4eb1-bfca-0a3cdbf3bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:59.198173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19716948-efe3-4eb1-bfca-0a3cdbf3bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:59.198468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19716948-efe3-4eb1-bfca-0a3cdbf3bfb8 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:59.199146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19716948-efe3-4eb1-bfca-0a3cdbf3bfb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:59.199452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19716948-efe3-4eb1-bfca-0a3cdbf3bfb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:59.200866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19716948-efe3-4eb1-bfca-0a3cdbf3bfb8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:59.219181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 341/668] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:59.219181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19716948-efe3-4eb1-bfca-0a3cdbf3bfb8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:59.251197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edfc10-51cb-4a76-aa4f-0130bae1a6bf None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:59.286563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63819b0-0b69-4194-9962-a8e0979e9b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:59.287206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63819b0-0b69-4194-9962-a8e0979e9b3c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:59.287487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63819b0-0b69-4194-9962-a8e0979e9b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:59.287758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63819b0-0b69-4194-9962-a8e0979e9b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:59.293283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:59.309207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:59.326571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19716948-efe3-4eb1-bfca-0a3cdbf3bfb8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:59.356193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63819b0-0b69-4194-9962-a8e0979e9b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:59.357738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63819b0-0b69-4194-9962-a8e0979e9b3c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:59.361325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87f164-344c-4354-8d8f-568c894b95d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:59.361977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f164-344c-4354-8d8f-568c894b95d4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:59.362253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f164-344c-4354-8d8f-568c894b95d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:59.362524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f164-344c-4354-8d8f-568c894b95d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:59.363202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87f164-344c-4354-8d8f-568c894b95d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:59.363498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87f164-344c-4354-8d8f-568c894b95d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:59.364847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87f164-344c-4354-8d8f-568c894b95d4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:59.377793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 342/669] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:46:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:59.377793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87f164-344c-4354-8d8f-568c894b95d4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:59.484104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 328/670] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:46:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:46:59.484104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63819b0-0b69-4194-9962-a8e0979e9b3c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:59.484488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87f164-344c-4354-8d8f-568c894b95d4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:59.519714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cd760-55af-4957-9067-f8606aa05c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:59.520326 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cd760-55af-4957-9067-f8606aa05c18 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:59.520607 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cd760-55af-4957-9067-f8606aa05c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:59.520880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cd760-55af-4957-9067-f8606aa05c18 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:59.521521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cd760-55af-4957-9067-f8606aa05c18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:59.521826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cd760-55af-4957-9067-f8606aa05c18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:59.523296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cd760-55af-4957-9067-f8606aa05c18 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:59.524820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:59.539851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 343/671] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:59.539851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cd760-55af-4957-9067-f8606aa05c18 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:59.540115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:59.586654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63819b0-0b69-4194-9962-a8e0979e9b3c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:59.619403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee9c37-7eaf-462a-bec6-ceebc10b5538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:59.620008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee9c37-7eaf-462a-bec6-ceebc10b5538 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:59.620288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee9c37-7eaf-462a-bec6-ceebc10b5538 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:59.620581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee9c37-7eaf-462a-bec6-ceebc10b5538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:59.621314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee9c37-7eaf-462a-bec6-ceebc10b5538 None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=e32b5bf1a2524f1cbc432f38195e4c08) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:46:59.621623 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee9c37-7eaf-462a-bec6-ceebc10b5538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:59.623102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee9c37-7eaf-462a-bec6-ceebc10b5538 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:59.628703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 329/672] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:46:59 2022] GET /identity/v3/role_assignments?user.id=e32b5bf1a2524f1cbc432f38195e4c08&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:46:59.629491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee9c37-7eaf-462a-bec6-ceebc10b5538 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:59.652890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cd760-55af-4957-9067-f8606aa05c18 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:59.702443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10992da6-9dd9-4365-94f7-7a6c745ccd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:59.704172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10992da6-9dd9-4365-94f7-7a6c745ccd08 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:59.705035 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10992da6-9dd9-4365-94f7-7a6c745ccd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:59.705576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10992da6-9dd9-4365-94f7-7a6c745ccd08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:59.723348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10992da6-9dd9-4365-94f7-7a6c745ccd08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:59.725155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10992da6-9dd9-4365-94f7-7a6c745ccd08 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-10992da6-9dd9-4365-94f7-7a6c745ccd08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:59.750034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:59.751578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:59.752746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:59.752746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:59.752746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:59.753668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee9c37-7eaf-462a-bec6-ceebc10b5538 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:59.757507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 344/673] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:59.760117 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10992da6-9dd9-4365-94f7-7a6c745ccd08 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:59.798926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96567a7b-dc99-45d9-ab1c-99c94ad5852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:59.800186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96567a7b-dc99-45d9-ab1c-99c94ad5852c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:59.800705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96567a7b-dc99-45d9-ab1c-99c94ad5852c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:59.801200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96567a7b-dc99-45d9-ab1c-99c94ad5852c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:59.815487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96567a7b-dc99-45d9-ab1c-99c94ad5852c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:59.817836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96567a7b-dc99-45d9-ab1c-99c94ad5852c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:59.819830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:59.828619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-96567a7b-dc99-45d9-ab1c-99c94ad5852c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:46:59.837237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:59.838279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:46:59.839264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:46:59.839264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:46:59.839264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:46:59.839264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 330/674] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:59.839493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96567a7b-dc99-45d9-ab1c-99c94ad5852c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:59.895143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10992da6-9dd9-4365-94f7-7a6c745ccd08 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:46:59.944923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd2b90-8f43-4cc7-a706-4e6047a6a8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:46:59.945576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd2b90-8f43-4cc7-a706-4e6047a6a8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:46:59.945854 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd2b90-8f43-4cc7-a706-4e6047a6a8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:46:59.946123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd2b90-8f43-4cc7-a706-4e6047a6a8d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:46:59.954223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd2b90-8f43-4cc7-a706-4e6047a6a8d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:46:59.956828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd2b90-8f43-4cc7-a706-4e6047a6a8d1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-27dd2b90-8f43-4cc7-a706-4e6047a6a8d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:46:59.971690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:46:59.972631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:46:59.973513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:46:59.973513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:46:59.973513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:46:59.973513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:46:59.973513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:46:59.973513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:46:59.973513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:46:59.973513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:46:59.974898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 345/675] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:46:59.975583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd2b90-8f43-4cc7-a706-4e6047a6a8d1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:46:59.980281 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96567a7b-dc99-45d9-ab1c-99c94ad5852c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:00.017813 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b95640-6dad-4cf3-9cf6-ca8b9531cb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:00.018433 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b95640-6dad-4cf3-9cf6-ca8b9531cb0a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:00.018739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b95640-6dad-4cf3-9cf6-ca8b9531cb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:00.019018 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b95640-6dad-4cf3-9cf6-ca8b9531cb0a None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:00.019669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b95640-6dad-4cf3-9cf6-ca8b9531cb0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:00.019975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b95640-6dad-4cf3-9cf6-ca8b9531cb0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:00.021544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b95640-6dad-4cf3-9cf6-ca8b9531cb0a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:00.035118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 331/676] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:00.035848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b95640-6dad-4cf3-9cf6-ca8b9531cb0a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:00.067880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:00.116394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd2b90-8f43-4cc7-a706-4e6047a6a8d1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:00.127725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:00.155783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b95640-6dad-4cf3-9cf6-ca8b9531cb0a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:00.156715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce69a4-67e6-4944-bf90-0170da49fc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:00.157477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce69a4-67e6-4944-bf90-0170da49fc9b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:00.157949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce69a4-67e6-4944-bf90-0170da49fc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:00.158389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce69a4-67e6-4944-bf90-0170da49fc9b None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:00.159812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce69a4-67e6-4944-bf90-0170da49fc9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:00.159812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce69a4-67e6-4944-bf90-0170da49fc9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:00.161163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce69a4-67e6-4944-bf90-0170da49fc9b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:00.176118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 346/677] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:00.177173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce69a4-67e6-4944-bf90-0170da49fc9b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:00.199818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429716d5-84b7-45cb-888a-79858df8caec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:00.201124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429716d5-84b7-45cb-888a-79858df8caec None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:00.201577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429716d5-84b7-45cb-888a-79858df8caec None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:00.202258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429716d5-84b7-45cb-888a-79858df8caec None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:00.203148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429716d5-84b7-45cb-888a-79858df8caec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:00.203777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429716d5-84b7-45cb-888a-79858df8caec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:00.206105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429716d5-84b7-45cb-888a-79858df8caec None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:00.226809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 332/678] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:00.227009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429716d5-84b7-45cb-888a-79858df8caec None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:00.292263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce69a4-67e6-4944-bf90-0170da49fc9b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:00.327074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30659527-854c-4dbc-a226-e36d63c3efe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:00.327734 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30659527-854c-4dbc-a226-e36d63c3efe8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:00.328011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30659527-854c-4dbc-a226-e36d63c3efe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:00.328285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30659527-854c-4dbc-a226-e36d63c3efe8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:00.336540 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30659527-854c-4dbc-a226-e36d63c3efe8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:00.338570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30659527-854c-4dbc-a226-e36d63c3efe8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:00.339584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-30659527-854c-4dbc-a226-e36d63c3efe8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:00.355381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:00.356448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:00.357403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:00.357403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:00.357403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:00.357403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:00.357403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:00.357403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:00.357403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:00.357403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:00.357403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:47:00.357403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:00.357403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 347/679] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:00.358038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30659527-854c-4dbc-a226-e36d63c3efe8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:00.363602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429716d5-84b7-45cb-888a-79858df8caec None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:00.387994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:00.410410 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e775ec-021e-4935-a882-261019730bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:00.411083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e775ec-021e-4935-a882-261019730bcd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:00.411366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e775ec-021e-4935-a882-261019730bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:00.411643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e775ec-021e-4935-a882-261019730bcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:00.420984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e775ec-021e-4935-a882-261019730bcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:00.422714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e775ec-021e-4935-a882-261019730bcd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-19e775ec-021e-4935-a882-261019730bcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:00.435153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:00.436109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:00.437076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:00.437076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:00.437076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:00.437076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:00.437076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:00.437076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:00.437076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:00.437076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:00.437076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 333/680] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:00.437851 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e775ec-021e-4935-a882-261019730bcd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:00.477619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30659527-854c-4dbc-a226-e36d63c3efe8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:00.513796 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05d586-186a-4b84-b8b0-1605a1594346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:00.514823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05d586-186a-4b84-b8b0-1605a1594346 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:00.515308 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05d586-186a-4b84-b8b0-1605a1594346 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:00.516033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05d586-186a-4b84-b8b0-1605a1594346 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:00.517881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05d586-186a-4b84-b8b0-1605a1594346 None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=fb514134abe54e5ca8125e572a2ff928) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:00.518420 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05d586-186a-4b84-b8b0-1605a1594346 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:00.520106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05d586-186a-4b84-b8b0-1605a1594346 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:00.534721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 348/681] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:00 2022] GET /identity/v3/role_assignments?user.id=fb514134abe54e5ca8125e572a2ff928&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:00.534721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05d586-186a-4b84-b8b0-1605a1594346 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:00.556494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e775ec-021e-4935-a882-261019730bcd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:00.599223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3666b1-f0a5-41ea-93f5-71fd2343e7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:00.600001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3666b1-f0a5-41ea-93f5-71fd2343e7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:00.600385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3666b1-f0a5-41ea-93f5-71fd2343e7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:00.600762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3666b1-f0a5-41ea-93f5-71fd2343e7b3 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:00.601607 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3666b1-f0a5-41ea-93f5-71fd2343e7b3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:00.601986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3666b1-f0a5-41ea-93f5-71fd2343e7b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:00.603625 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3666b1-f0a5-41ea-93f5-71fd2343e7b3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:00.624455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 334/682] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Sep 16 07:47:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:00.625969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4e89e-0da3-496c-8774-a7f6adf6b92d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:00.626323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4e89e-0da3-496c-8774-a7f6adf6b92d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:00.626673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4e89e-0da3-496c-8774-a7f6adf6b92d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:00.628357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 335/683] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:00.637768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730902-ed0a-409f-a3ca-0c4e0d83dc7d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:00.638210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730902-ed0a-409f-a3ca-0c4e0d83dc7d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:00.638430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730902-ed0a-409f-a3ca-0c4e0d83dc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:00.653099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:00.653738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:00.664098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e1730902-ed0a-409f-a3ca-0c4e0d83dc7d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:00.665368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05d586-186a-4b84-b8b0-1605a1594346 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:00.666803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e1730902-ed0a-409f-a3ca-0c4e0d83dc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:00.710407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15436d-8161-4029-948f-1cdd6eb6946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:00.711370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15436d-8161-4029-948f-1cdd6eb6946b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:00.711484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15436d-8161-4029-948f-1cdd6eb6946b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:00.711801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15436d-8161-4029-948f-1cdd6eb6946b None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:00.712619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15436d-8161-4029-948f-1cdd6eb6946b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:00.713268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15436d-8161-4029-948f-1cdd6eb6946b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:00.715276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15436d-8161-4029-948f-1cdd6eb6946b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:00.731720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 349/684] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:00.732518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15436d-8161-4029-948f-1cdd6eb6946b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:00.775904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 336/685] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:47:00.777052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730902-ed0a-409f-a3ca-0c4e0d83dc7d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:00.863045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15436d-8161-4029-948f-1cdd6eb6946b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:00.908874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3bb81-7763-4552-9809-25f095086a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:00.909787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3bb81-7763-4552-9809-25f095086a63 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:00.910298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3bb81-7763-4552-9809-25f095086a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:00.910837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3bb81-7763-4552-9809-25f095086a63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:00.943862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:00.945357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730902-ed0a-409f-a3ca-0c4e0d83dc7d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:01.008631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3bb81-7763-4552-9809-25f095086a63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:01.010581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3bb81-7763-4552-9809-25f095086a63 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:01.020724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0155b1-eb17-431d-9bb8-107494151b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:01.021344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0155b1-eb17-431d-9bb8-107494151b91 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:01.021622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0155b1-eb17-431d-9bb8-107494151b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:01.021895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0155b1-eb17-431d-9bb8-107494151b91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:01.022628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0155b1-eb17-431d-9bb8-107494151b91 None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=a29c4e53ac9a438e9701a70f3de3f32e) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:01.022959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0155b1-eb17-431d-9bb8-107494151b91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:01.024375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0155b1-eb17-431d-9bb8-107494151b91 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:01.036448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:01.042568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 337/686] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:00 2022] GET /identity/v3/role_assignments?user.id=a29c4e53ac9a438e9701a70f3de3f32e&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:01.081673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0155b1-eb17-431d-9bb8-107494151b91 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:01.154702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 350/687] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:47:01.160620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3bb81-7763-4552-9809-25f095086a63 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:01.259105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:01.263377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0155b1-eb17-431d-9bb8-107494151b91 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:01.309043 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93cbe-0693-44b4-accb-c02703ca68b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:01.310216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93cbe-0693-44b4-accb-c02703ca68b7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:01.310667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93cbe-0693-44b4-accb-c02703ca68b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:01.311153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93cbe-0693-44b4-accb-c02703ca68b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:01.312152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3bb81-7763-4552-9809-25f095086a63 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:01.325250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb93cbe-0693-44b4-accb-c02703ca68b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:01.327000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb93cbe-0693-44b4-accb-c02703ca68b7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-acb93cbe-0693-44b4-accb-c02703ca68b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:01.345931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:01.347425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:01.349835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:01.349835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:01.349835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:01.349835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 338/688] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:01.354923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93cbe-0693-44b4-accb-c02703ca68b7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:01.358086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a69734-f863-4593-b12f-0e83e56bd7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:01.359023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a69734-f863-4593-b12f-0e83e56bd7ea None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:01.359512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a69734-f863-4593-b12f-0e83e56bd7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:01.359989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a69734-f863-4593-b12f-0e83e56bd7ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:01.371610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a69734-f863-4593-b12f-0e83e56bd7ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:01.374131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a69734-f863-4593-b12f-0e83e56bd7ea None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-35a69734-f863-4593-b12f-0e83e56bd7ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:01.394436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:01.395505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:01.396427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:01.396427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:01.396427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:01.398973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 351/689] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:01.404846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a69734-f863-4593-b12f-0e83e56bd7ea None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:01.416883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:01.481739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93cbe-0693-44b4-accb-c02703ca68b7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:01.517268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a69734-f863-4593-b12f-0e83e56bd7ea None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:01.518890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:01.526775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f54294-05d1-4f0b-8613-7e324ecec764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:01.532593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f54294-05d1-4f0b-8613-7e324ecec764 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:01.532593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f54294-05d1-4f0b-8613-7e324ecec764 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:01.532593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f54294-05d1-4f0b-8613-7e324ecec764 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:01.532593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f54294-05d1-4f0b-8613-7e324ecec764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:01.532593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f54294-05d1-4f0b-8613-7e324ecec764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:01.532593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f54294-05d1-4f0b-8613-7e324ecec764 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:01.549130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 339/690] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:01.554793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f54294-05d1-4f0b-8613-7e324ecec764 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:01.563094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32206053-37d2-459f-82d2-46cbaff843e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:01.563752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32206053-37d2-459f-82d2-46cbaff843e9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:01.564039 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32206053-37d2-459f-82d2-46cbaff843e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:01.564315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32206053-37d2-459f-82d2-46cbaff843e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:01.564959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32206053-37d2-459f-82d2-46cbaff843e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:01.565262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32206053-37d2-459f-82d2-46cbaff843e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:01.566706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32206053-37d2-459f-82d2-46cbaff843e9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:01.588143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 352/691] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:01.590157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32206053-37d2-459f-82d2-46cbaff843e9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:01.762930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32206053-37d2-459f-82d2-46cbaff843e9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:01.763685 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f54294-05d1-4f0b-8613-7e324ecec764 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:01.788841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:01.805268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adecfc-c41f-4d81-929e-593653569b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:01.806352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adecfc-c41f-4d81-929e-593653569b1b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:01.806881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adecfc-c41f-4d81-929e-593653569b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:01.807497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adecfc-c41f-4d81-929e-593653569b1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:01.815900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adecfc-c41f-4d81-929e-593653569b1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:01.818116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adecfc-c41f-4d81-929e-593653569b1b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-94adecfc-c41f-4d81-929e-593653569b1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:01.836417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:01.838000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:01.838929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:01.838929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:01.838929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:01.838929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:01.838929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:01.838929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:01.838929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:01.838929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:01.841482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65f954-8536-4428-b597-b1e9757e2d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:01.842334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65f954-8536-4428-b597-b1e9757e2d91 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:01.842948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65f954-8536-4428-b597-b1e9757e2d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:01.843443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65f954-8536-4428-b597-b1e9757e2d91 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:01.844314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65f954-8536-4428-b597-b1e9757e2d91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:01.845951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65f954-8536-4428-b597-b1e9757e2d91 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:01.849707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 353/692] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:01.855789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adecfc-c41f-4d81-929e-593653569b1b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:01.885270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:01.968212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adecfc-c41f-4d81-929e-593653569b1b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:02.003838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be589418-c30c-4959-bd70-6114ccfb62d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:02.004544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be589418-c30c-4959-bd70-6114ccfb62d3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:02.004923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be589418-c30c-4959-bd70-6114ccfb62d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:02.005300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be589418-c30c-4959-bd70-6114ccfb62d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:02.006181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be589418-c30c-4959-bd70-6114ccfb62d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:02.006589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be589418-c30c-4959-bd70-6114ccfb62d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:02.008130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be589418-c30c-4959-bd70-6114ccfb62d3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:02.022253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 354/693] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:02.029056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be589418-c30c-4959-bd70-6114ccfb62d3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:02.030238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 340/694] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:47:01 2022] POST /identity/v3/users => generated 254 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:47:02.144160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:02.167111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be589418-c30c-4959-bd70-6114ccfb62d3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:02.187002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d07292-d7b6-4d49-81b3-26a6aada3ba4 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:02.187871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d07292-d7b6-4d49-81b3-26a6aada3ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:02.188371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d07292-d7b6-4d49-81b3-26a6aada3ba4 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:02.190218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 341/695] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:02.196293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4dc7-a4cf-4eaf-b33d-1beaef274d4e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:02.197427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4dc7-a4cf-4eaf-b33d-1beaef274d4e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:02.197902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de4dc7-a4cf-4eaf-b33d-1beaef274d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:02.211373 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a32988-946c-44c7-92d0-8349000c7f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:02.212119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a32988-946c-44c7-92d0-8349000c7f8c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:02.212985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a32988-946c-44c7-92d0-8349000c7f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:02.213322 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a32988-946c-44c7-92d0-8349000c7f8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:02.214267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a32988-946c-44c7-92d0-8349000c7f8c None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=a1ae39fdd6ff4841843c9826dd88229a) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:02.214643 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a32988-946c-44c7-92d0-8349000c7f8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:02.216370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a32988-946c-44c7-92d0-8349000c7f8c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:02.226714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 355/696] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:02 2022] GET /identity/v3/role_assignments?user.id=a1ae39fdd6ff4841843c9826dd88229a&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:02.232885 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-85de4dc7-a4cf-4eaf-b33d-1beaef274d4e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:02.234426 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-85de4dc7-a4cf-4eaf-b33d-1beaef274d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:02.315553 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 342/697] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 07:47:02.591503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a32988-946c-44c7-92d0-8349000c7f8c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:02.699938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a32988-946c-44c7-92d0-8349000c7f8c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:02.743491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372b362-345c-4b3b-ace7-f19af2b05939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:02.744155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372b362-345c-4b3b-ace7-f19af2b05939 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:02.744441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372b362-345c-4b3b-ace7-f19af2b05939 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:02.744716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372b362-345c-4b3b-ace7-f19af2b05939 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:02.759661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372b362-345c-4b3b-ace7-f19af2b05939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-1372b362-345c-4b3b-ace7-f19af2b05939 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:02.765002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:02.765954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:02.766837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:02.769728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 356/698] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:02.771791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de4dc7-a4cf-4eaf-b33d-1beaef274d4e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:02.919849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:02.923956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de4dc7-a4cf-4eaf-b33d-1beaef274d4e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:02.971742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9eeeeb-aec7-4791-9c8b-2ddc949f5310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:02.972359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eeeeb-aec7-4791-9c8b-2ddc949f5310 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:02.972684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eeeeb-aec7-4791-9c8b-2ddc949f5310 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:02.972955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eeeeb-aec7-4791-9c8b-2ddc949f5310 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:02.973666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9eeeeb-aec7-4791-9c8b-2ddc949f5310 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:02.973971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9eeeeb-aec7-4791-9c8b-2ddc949f5310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:02.975367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9eeeeb-aec7-4791-9c8b-2ddc949f5310 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:02.984720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 343/699] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:47:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:02.991281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372b362-345c-4b3b-ace7-f19af2b05939 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:03.004729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:03.145887 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372b362-345c-4b3b-ace7-f19af2b05939 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:03.194849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0f984-55c2-4c4f-9285-357c96dfe775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:03.195611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0f984-55c2-4c4f-9285-357c96dfe775 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:03.195773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0f984-55c2-4c4f-9285-357c96dfe775 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:03.196018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0f984-55c2-4c4f-9285-357c96dfe775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:03.210306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0f984-55c2-4c4f-9285-357c96dfe775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:03.211947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0f984-55c2-4c4f-9285-357c96dfe775 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-44a0f984-55c2-4c4f-9285-357c96dfe775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:03.230852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:03.232992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:03.235090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:03.235090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:03.235090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:03.235090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 357/700] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:03.244296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9eeeeb-aec7-4791-9c8b-2ddc949f5310 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:03.323803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048d032-1fcc-47cf-a80f-1f8a2069a5f9 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:03.324103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048d032-1fcc-47cf-a80f-1f8a2069a5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:03.324366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048d032-1fcc-47cf-a80f-1f8a2069a5f9 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:03.325932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 358/701] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:03.337601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5add9-e408-4915-b5db-28f22e7c406d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:03.337966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5add9-e408-4915-b5db-28f22e7c406d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:03.338333 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5add9-e408-4915-b5db-28f22e7c406d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:03.375684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f5f5add9-e408-4915-b5db-28f22e7c406d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:03.377004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f5f5add9-e408-4915-b5db-28f22e7c406d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:03.389606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9eeeeb-aec7-4791-9c8b-2ddc949f5310 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:03.436347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f8cdf-fd18-47cf-9fc2-fdb1dbb80c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:03.437680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f8cdf-fd18-47cf-9fc2-fdb1dbb80c52 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:03.438179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f8cdf-fd18-47cf-9fc2-fdb1dbb80c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:03.438716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f8cdf-fd18-47cf-9fc2-fdb1dbb80c52 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:03.439648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f8cdf-fd18-47cf-9fc2-fdb1dbb80c52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:03.440146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f8cdf-fd18-47cf-9fc2-fdb1dbb80c52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:03.441702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f8cdf-fd18-47cf-9fc2-fdb1dbb80c52 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:03.455651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 344/702] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:03.460887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f8cdf-fd18-47cf-9fc2-fdb1dbb80c52 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:03.467655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:03.492137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:03.498837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 359/703] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:47:03.615311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f8cdf-fd18-47cf-9fc2-fdb1dbb80c52 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:03.672449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd27550-5759-4656-9f93-d9b73293409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:03.673519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd27550-5759-4656-9f93-d9b73293409d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:03.673993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd27550-5759-4656-9f93-d9b73293409d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:03.674534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd27550-5759-4656-9f93-d9b73293409d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:03.686175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd27550-5759-4656-9f93-d9b73293409d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:03.688212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd27550-5759-4656-9f93-d9b73293409d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3bd27550-5759-4656-9f93-d9b73293409d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:03.704124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:03.705414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:03.706396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:03.706396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:03.706396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:03.706396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:03.706396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:03.706396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:03.706396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:03.706396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:03.706396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:03.706396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:03.708475 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 345/704] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Sep 16 07:47:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:03.715995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5add9-e408-4915-b5db-28f22e7c406d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:03.796210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd27550-5759-4656-9f93-d9b73293409d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:03.828369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5add9-e408-4915-b5db-28f22e7c406d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:03.868157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc27018-ba19-4cfd-a0d4-c715d548550b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:03.868831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc27018-ba19-4cfd-a0d4-c715d548550b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:03.869108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc27018-ba19-4cfd-a0d4-c715d548550b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:03.869379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc27018-ba19-4cfd-a0d4-c715d548550b None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:03.870119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc27018-ba19-4cfd-a0d4-c715d548550b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:03.870423 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc27018-ba19-4cfd-a0d4-c715d548550b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:03.872448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc27018-ba19-4cfd-a0d4-c715d548550b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:03.893023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 360/705] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Sep 16 07:47:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:03.898481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc27018-ba19-4cfd-a0d4-c715d548550b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:03.968718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:03.984017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:03.997472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd27550-5759-4656-9f93-d9b73293409d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:04.039118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc27018-ba19-4cfd-a0d4-c715d548550b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:04.043263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fccfa-53c7-4447-acc8-d4280c295d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:04.043903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fccfa-53c7-4447-acc8-d4280c295d35 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:04.044178 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fccfa-53c7-4447-acc8-d4280c295d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:04.044449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fccfa-53c7-4447-acc8-d4280c295d35 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:04.045031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fccfa-53c7-4447-acc8-d4280c295d35 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:04.046392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fccfa-53c7-4447-acc8-d4280c295d35 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0b5fccfa-53c7-4447-acc8-d4280c295d35 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:04.060860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:04.063277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:04.065029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 346/706] 10.222.0.59 () {56 vars in 1129 bytes} [Fri Sep 16 07:47:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:04.066426 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09afe4-0000-47ce-9c02-7fbf97919d17 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:04.066426 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09afe4-0000-47ce-9c02-7fbf97919d17 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:04.066426 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09afe4-0000-47ce-9c02-7fbf97919d17 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:04.066426 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 347/707] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:04.074701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09afe4-0000-47ce-9c02-7fbf97919d17 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:04.081279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a60c6f-2531-477d-8c0b-f7113954d791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:04.081991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a60c6f-2531-477d-8c0b-f7113954d791 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:04.082341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a60c6f-2531-477d-8c0b-f7113954d791 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:04.082740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a60c6f-2531-477d-8c0b-f7113954d791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:04.166015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a60c6f-2531-477d-8c0b-f7113954d791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:04.167919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a60c6f-2531-477d-8c0b-f7113954d791 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:04.183840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09afe4-0000-47ce-9c02-7fbf97919d17 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:04.222232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:04.222764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25546d-85a6-4ec5-b830-e2b68b72b691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:04.224677 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25546d-85a6-4ec5-b830-e2b68b72b691 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:04.225028 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25546d-85a6-4ec5-b830-e2b68b72b691 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:04.225374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25546d-85a6-4ec5-b830-e2b68b72b691 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:04.226080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25546d-85a6-4ec5-b830-e2b68b72b691 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:04.226610 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25546d-85a6-4ec5-b830-e2b68b72b691 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:04.229084 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25546d-85a6-4ec5-b830-e2b68b72b691 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:04.235369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 348/708] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Sep 16 07:47:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:04.236993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfc572-eb3f-4e76-9a56-66e30a7e7687 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:04.237263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfc572-eb3f-4e76-9a56-66e30a7e7687 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:04.237613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfc572-eb3f-4e76-9a56-66e30a7e7687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:04.261960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dabfc572-eb3f-4e76-9a56-66e30a7e7687 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:04.263596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dabfc572-eb3f-4e76-9a56-66e30a7e7687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:04.266412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:04.288874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 361/709] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 07:47:04.289745 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a60c6f-2531-477d-8c0b-f7113954d791 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:04.362839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 349/710] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:47:04.363791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfc572-eb3f-4e76-9a56-66e30a7e7687 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:04.435798 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a60c6f-2531-477d-8c0b-f7113954d791 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:04.502954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c29dc-4fdd-43ed-ac8c-4cdc798b18ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:04.503492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c29dc-4fdd-43ed-ac8c-4cdc798b18ca None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:04.503773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c29dc-4fdd-43ed-ac8c-4cdc798b18ca None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:04.504048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c29dc-4fdd-43ed-ac8c-4cdc798b18ca None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:04.504692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c29dc-4fdd-43ed-ac8c-4cdc798b18ca None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:04.504999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c29dc-4fdd-43ed-ac8c-4cdc798b18ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:04.506368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c29dc-4fdd-43ed-ac8c-4cdc798b18ca None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:04.518169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 362/711] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Sep 16 07:47:04 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:04.537807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfc572-eb3f-4e76-9a56-66e30a7e7687 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:04.575856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19759fa5-cd0e-47a0-b6f8-a41fe7de8657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:04.576591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19759fa5-cd0e-47a0-b6f8-a41fe7de8657 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:04.576874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19759fa5-cd0e-47a0-b6f8-a41fe7de8657 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:04.577150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19759fa5-cd0e-47a0-b6f8-a41fe7de8657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:04.584215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:04.588613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19759fa5-cd0e-47a0-b6f8-a41fe7de8657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:04.590141 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19759fa5-cd0e-47a0-b6f8-a41fe7de8657 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-19759fa5-cd0e-47a0-b6f8-a41fe7de8657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:04.606395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:04.607437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:04.608361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:04.608361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:04.608361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:04.608361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 350/712] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:04.619823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c29dc-4fdd-43ed-ac8c-4cdc798b18ca None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:04.635089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:04.696302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19759fa5-cd0e-47a0-b6f8-a41fe7de8657 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:04.807177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c29dc-4fdd-43ed-ac8c-4cdc798b18ca None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:04.858418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19759fa5-cd0e-47a0-b6f8-a41fe7de8657 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:04.869185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5312290-adee-48af-bf76-24f424d154a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:04.869812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5312290-adee-48af-bf76-24f424d154a1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:04.870503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5312290-adee-48af-bf76-24f424d154a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:04.870865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5312290-adee-48af-bf76-24f424d154a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:04.871653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5312290-adee-48af-bf76-24f424d154a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:04.872109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5312290-adee-48af-bf76-24f424d154a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:04.873661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5312290-adee-48af-bf76-24f424d154a1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:04.890575 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 363/713] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:04.895398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052b28d-716e-4a82-9b28-56c6060f93a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:04.896035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b28d-716e-4a82-9b28-56c6060f93a1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:04.896313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b28d-716e-4a82-9b28-56c6060f93a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:04.896420 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f86dce-cf17-40b5-9d81-33fb9f69dab4 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:04.896600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b28d-716e-4a82-9b28-56c6060f93a1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:04.896695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f86dce-cf17-40b5-9d81-33fb9f69dab4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:04.897006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f86dce-cf17-40b5-9d81-33fb9f69dab4 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:04.897170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052b28d-716e-4a82-9b28-56c6060f93a1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:04.898396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 364/714] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:04.898788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052b28d-716e-4a82-9b28-56c6060f93a1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:04.899159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f86dce-cf17-40b5-9d81-33fb9f69dab4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3052b28d-716e-4a82-9b28-56c6060f93a1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:04.912843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:04.913985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:04.916114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 351/715] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:04 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:04.916971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3515e-03f0-433d-b842-29268d2e80dd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:04.916971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3515e-03f0-433d-b842-29268d2e80dd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:04.916971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3515e-03f0-433d-b842-29268d2e80dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:04.980137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a4e3515e-03f0-433d-b842-29268d2e80dd None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:04.981638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a4e3515e-03f0-433d-b842-29268d2e80dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:04.983373 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:05.023219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f86dce-cf17-40b5-9d81-33fb9f69dab4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:05.044660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:05.064281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f9eef-45ef-480f-998f-0df908c913e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:05.070874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 352/716] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:47:05.070874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3515e-03f0-433d-b842-29268d2e80dd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:05.073289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f9eef-45ef-480f-998f-0df908c913e2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:05.073741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f9eef-45ef-480f-998f-0df908c913e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:05.074174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f9eef-45ef-480f-998f-0df908c913e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:05.085919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f9eef-45ef-480f-998f-0df908c913e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:05.088221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f9eef-45ef-480f-998f-0df908c913e2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b13f9eef-45ef-480f-998f-0df908c913e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:05.102278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:05.103290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:05.104177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:05.104177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:05.104177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:05.104177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:05.104177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:05.104177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:05.104177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:05.104177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:05.106784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 365/717] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:05.114722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f9eef-45ef-480f-998f-0df908c913e2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:05.208187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3515e-03f0-433d-b842-29268d2e80dd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:05.235215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f9eef-45ef-480f-998f-0df908c913e2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:05.249206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5b644-0baa-44cf-bda8-bae7e41ff21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:05.250141 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5b644-0baa-44cf-bda8-bae7e41ff21b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:05.250485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5b644-0baa-44cf-bda8-bae7e41ff21b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:05.251240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5b644-0baa-44cf-bda8-bae7e41ff21b None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:05.252143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5b644-0baa-44cf-bda8-bae7e41ff21b None admin] RBAC: Adding query filter params (name=network) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:05.252450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5b644-0baa-44cf-bda8-bae7e41ff21b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:05.254452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5b644-0baa-44cf-bda8-bae7e41ff21b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:05.260616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 353/718] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:05.262532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457d8e1-ca61-4c93-ac67-d35bb2624187 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:05.262964 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457d8e1-ca61-4c93-ac67-d35bb2624187 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:05.263238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457d8e1-ca61-4c93-ac67-d35bb2624187 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:05.268857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 354/719] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:05.269616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457d8e1-ca61-4c93-ac67-d35bb2624187 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:05.273801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:05.280440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a26de0-7b2d-4d0e-ab14-f67a8171fe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:05.284914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a26de0-7b2d-4d0e-ab14-f67a8171fe8c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:05.284914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a26de0-7b2d-4d0e-ab14-f67a8171fe8c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:05.284914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a26de0-7b2d-4d0e-ab14-f67a8171fe8c None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:05.284914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a26de0-7b2d-4d0e-ab14-f67a8171fe8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:05.284914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a26de0-7b2d-4d0e-ab14-f67a8171fe8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:05.284914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a26de0-7b2d-4d0e-ab14-f67a8171fe8c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:05.300804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 366/720] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:05.302827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d256757-e0fa-4461-9648-188344dca29b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:05.303307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d256757-e0fa-4461-9648-188344dca29b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:05.303759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d256757-e0fa-4461-9648-188344dca29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:05.325010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8d256757-e0fa-4461-9648-188344dca29b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:05.326574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8d256757-e0fa-4461-9648-188344dca29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:05.330135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:05.391832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457d8e1-ca61-4c93-ac67-d35bb2624187 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:05.406709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 367/721] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:47:05.406709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d256757-e0fa-4461-9648-188344dca29b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:05.435561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7678e75-7c45-4919-852a-45479b21a8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:05.436299 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7678e75-7c45-4919-852a-45479b21a8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:05.436586 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7678e75-7c45-4919-852a-45479b21a8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:05.436863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7678e75-7c45-4919-852a-45479b21a8f3 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:05.437752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7678e75-7c45-4919-852a-45479b21a8f3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:05.438314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7678e75-7c45-4919-852a-45479b21a8f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:05.440651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7678e75-7c45-4919-852a-45479b21a8f3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:05.448973 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 355/722] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:05.449807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7678e75-7c45-4919-852a-45479b21a8f3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:05.554958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d256757-e0fa-4461-9648-188344dca29b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:05.580813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:05.597389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456efd0-4bc2-4721-96e0-bbdf6669dab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:05.598430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456efd0-4bc2-4721-96e0-bbdf6669dab6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:05.598977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456efd0-4bc2-4721-96e0-bbdf6669dab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:05.599561 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456efd0-4bc2-4721-96e0-bbdf6669dab6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:05.605298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456efd0-4bc2-4721-96e0-bbdf6669dab6 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=e32b5bf1a2524f1cbc432f38195e4c08) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:05.605298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456efd0-4bc2-4721-96e0-bbdf6669dab6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:05.605298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456efd0-4bc2-4721-96e0-bbdf6669dab6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:05.610183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 368/723] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:05 2022] GET /identity/v3/role_assignments?user.id=e32b5bf1a2524f1cbc432f38195e4c08&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=62eb3e525077429aacc0ada63300802e => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:05.678216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7678e75-7c45-4919-852a-45479b21a8f3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:05.702663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:05.729008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e8db4-73ff-47e4-b01c-7268a811885a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:05.729660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e8db4-73ff-47e4-b01c-7268a811885a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:05.729939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e8db4-73ff-47e4-b01c-7268a811885a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:05.730212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e8db4-73ff-47e4-b01c-7268a811885a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:05.737713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456efd0-4bc2-4721-96e0-bbdf6669dab6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:05.742650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e8db4-73ff-47e4-b01c-7268a811885a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-196e8db4-73ff-47e4-b01c-7268a811885a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:05.748102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:05.749056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:05.749888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 356/724] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:47:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:05.755242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e8db4-73ff-47e4-b01c-7268a811885a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:05.897769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e8db4-73ff-47e4-b01c-7268a811885a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:05.949820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37f1b1-3bb2-4710-b0e4-704ffeda1a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:05.950513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37f1b1-3bb2-4710-b0e4-704ffeda1a5b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:05.950815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37f1b1-3bb2-4710-b0e4-704ffeda1a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:05.951085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37f1b1-3bb2-4710-b0e4-704ffeda1a5b None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:05.951755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37f1b1-3bb2-4710-b0e4-704ffeda1a5b None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:05.952055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37f1b1-3bb2-4710-b0e4-704ffeda1a5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:05.956121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37f1b1-3bb2-4710-b0e4-704ffeda1a5b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:05.960995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456efd0-4bc2-4721-96e0-bbdf6669dab6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:05.978637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 357/725] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:47:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:05.983837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37f1b1-3bb2-4710-b0e4-704ffeda1a5b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:06.022306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:06.049028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f7569-a3f3-4d8b-8187-c940833427bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:06.049683 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f7569-a3f3-4d8b-8187-c940833427bb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:06.049965 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f7569-a3f3-4d8b-8187-c940833427bb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:06.050240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f7569-a3f3-4d8b-8187-c940833427bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:06.085173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f7569-a3f3-4d8b-8187-c940833427bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-746f7569-a3f3-4d8b-8187-c940833427bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:06.092152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:06.093114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:06.093957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 369/726] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:06.099550 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f7569-a3f3-4d8b-8187-c940833427bb None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:06.108205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37f1b1-3bb2-4710-b0e4-704ffeda1a5b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:06.164006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:06.208300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8318b-efb4-4437-a3fb-7417ff2f48ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:06.208949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8318b-efb4-4437-a3fb-7417ff2f48ed None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:06.209229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8318b-efb4-4437-a3fb-7417ff2f48ed None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:06.209499 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8318b-efb4-4437-a3fb-7417ff2f48ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:06.219890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f7569-a3f3-4d8b-8187-c940833427bb None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:06.239696 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8318b-efb4-4437-a3fb-7417ff2f48ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:06.241277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8318b-efb4-4437-a3fb-7417ff2f48ed None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:06.255105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace090e-ec19-489e-8348-82747634d3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:06.255712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace090e-ec19-489e-8348-82747634d3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:06.255988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace090e-ec19-489e-8348-82747634d3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:06.256259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace090e-ec19-489e-8348-82747634d3d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:06.256919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace090e-ec19-489e-8348-82747634d3d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:06.257220 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace090e-ec19-489e-8348-82747634d3d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:06.258572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace090e-ec19-489e-8348-82747634d3d2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:06.268551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 370/727] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:47:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-43c8318b-efb4-4437-a3fb-7417ff2f48ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:06.269165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:06.270096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:06.271494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:06.271494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:06.271494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:06.273263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 358/728] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:06.273739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace090e-ec19-489e-8348-82747634d3d2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:06.279454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8318b-efb4-4437-a3fb-7417ff2f48ed None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:06.377338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace090e-ec19-489e-8348-82747634d3d2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:06.384709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:06.413607 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f2d67-e105-4610-8c27-59c83c233117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:06.414306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f2d67-e105-4610-8c27-59c83c233117 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:06.414524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f2d67-e105-4610-8c27-59c83c233117 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:06.414886 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f2d67-e105-4610-8c27-59c83c233117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:06.426593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f2d67-e105-4610-8c27-59c83c233117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:06.428065 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f2d67-e105-4610-8c27-59c83c233117 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-f10f2d67-e105-4610-8c27-59c83c233117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:06.443600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:06.444597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:06.445506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:06.445506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:06.445506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:06.445506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 371/729] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:06.454697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f2d67-e105-4610-8c27-59c83c233117 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:06.510960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:06.537799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8318b-efb4-4437-a3fb-7417ff2f48ed None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:06.569871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f2d67-e105-4610-8c27-59c83c233117 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:06.607231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0522db7-97d1-4adf-8b40-1eeaf5792e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:06.607842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0522db7-97d1-4adf-8b40-1eeaf5792e64 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:06.608174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0522db7-97d1-4adf-8b40-1eeaf5792e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:06.608453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0522db7-97d1-4adf-8b40-1eeaf5792e64 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:06.609087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0522db7-97d1-4adf-8b40-1eeaf5792e64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:06.609498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0522db7-97d1-4adf-8b40-1eeaf5792e64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:06.610886 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0522db7-97d1-4adf-8b40-1eeaf5792e64 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:06.619173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d8cf7-b468-4b1d-851d-41522b12fd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:06.619909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d8cf7-b468-4b1d-851d-41522b12fd0d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:06.620201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d8cf7-b468-4b1d-851d-41522b12fd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:06.620477 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d8cf7-b468-4b1d-851d-41522b12fd0d None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:06.621130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d8cf7-b468-4b1d-851d-41522b12fd0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:06.621436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d8cf7-b468-4b1d-851d-41522b12fd0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:06.622847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d8cf7-b468-4b1d-851d-41522b12fd0d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:06.623326 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 372/730] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:06.635891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0522db7-97d1-4adf-8b40-1eeaf5792e64 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:06.653233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 359/731] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:06.658548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d8cf7-b468-4b1d-851d-41522b12fd0d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:06.659098 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:06.737917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0522db7-97d1-4adf-8b40-1eeaf5792e64 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:06.789126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43540f0d-c687-464b-a884-e8bcf312b215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:06.789814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43540f0d-c687-464b-a884-e8bcf312b215 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:06.790184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43540f0d-c687-464b-a884-e8bcf312b215 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:06.791217 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43540f0d-c687-464b-a884-e8bcf312b215 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:06.801256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43540f0d-c687-464b-a884-e8bcf312b215 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:06.802720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43540f0d-c687-464b-a884-e8bcf312b215 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-43540f0d-c687-464b-a884-e8bcf312b215 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:06.827822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:06.828824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:06.829768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:06.829768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:06.829768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:06.829768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:06.829768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:06.829768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:06.829768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:06.829768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:06.829768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:47:06.829768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:06.829768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 373/732] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:06.836299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43540f0d-c687-464b-a884-e8bcf312b215 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:06.872818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d8cf7-b468-4b1d-851d-41522b12fd0d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:06.932784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8eb5cc-1800-4b63-930e-729069c444b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:06.934213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8eb5cc-1800-4b63-930e-729069c444b8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:06.934717 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8eb5cc-1800-4b63-930e-729069c444b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:06.935247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8eb5cc-1800-4b63-930e-729069c444b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:06.950141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:06.951979 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8eb5cc-1800-4b63-930e-729069c444b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:06.953449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8eb5cc-1800-4b63-930e-729069c444b8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:06.973041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43540f0d-c687-464b-a884-e8bcf312b215 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-de8eb5cc-1800-4b63-930e-729069c444b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:06.974532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:06.975534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:06.976482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:06.976482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:06.976482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:06.976482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:06.976482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:06.976482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:06.976482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:06.976482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:06.976482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:06.976482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:06.976482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 360/733] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Sep 16 07:47:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:06.980025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8eb5cc-1800-4b63-930e-729069c444b8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:06.994479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:07.013899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5939bbd-d2fc-482e-b594-04d1673b9989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:07.014850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5939bbd-d2fc-482e-b594-04d1673b9989 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:07.015324 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5939bbd-d2fc-482e-b594-04d1673b9989 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:07.015788 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5939bbd-d2fc-482e-b594-04d1673b9989 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:07.016687 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5939bbd-d2fc-482e-b594-04d1673b9989 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:07.017214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5939bbd-d2fc-482e-b594-04d1673b9989 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:07.018917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5939bbd-d2fc-482e-b594-04d1673b9989 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:07.037104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 374/734] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Sep 16 07:47:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:07.046709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5939bbd-d2fc-482e-b594-04d1673b9989 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:07.158329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8eb5cc-1800-4b63-930e-729069c444b8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:07.212881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c5b58-ed46-4f67-a01c-2804238eeb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:07.213499 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c5b58-ed46-4f67-a01c-2804238eeb9c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:07.213780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c5b58-ed46-4f67-a01c-2804238eeb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:07.214053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c5b58-ed46-4f67-a01c-2804238eeb9c None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:07.214811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c5b58-ed46-4f67-a01c-2804238eeb9c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:07.215119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c5b58-ed46-4f67-a01c-2804238eeb9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:07.216568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c5b58-ed46-4f67-a01c-2804238eeb9c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:07.245813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5939bbd-d2fc-482e-b594-04d1673b9989 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:07.251027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 361/735] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Sep 16 07:47:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:07.252376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d961-9f6f-4757-b426-e516e5eaecfb None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:07.252673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d961-9f6f-4757-b426-e516e5eaecfb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:07.252949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d961-9f6f-4757-b426-e516e5eaecfb None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:07.254144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 362/736] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:07.258824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86d961-9f6f-4757-b426-e516e5eaecfb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:07.283675 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e937c-8852-4c48-8bf8-b7f22d1e8592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:07.284398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e937c-8852-4c48-8bf8-b7f22d1e8592 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:07.284679 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e937c-8852-4c48-8bf8-b7f22d1e8592 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:07.284947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e937c-8852-4c48-8bf8-b7f22d1e8592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:07.289821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:07.334129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:07.368277 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e937c-8852-4c48-8bf8-b7f22d1e8592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:07.370095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e937c-8852-4c48-8bf8-b7f22d1e8592 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:07.372402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86d961-9f6f-4757-b426-e516e5eaecfb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:07.413385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba1bdf-fea4-4e26-80d9-b7501e16c1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:07.413995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba1bdf-fea4-4e26-80d9-b7501e16c1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:07.414276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba1bdf-fea4-4e26-80d9-b7501e16c1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:07.414550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba1bdf-fea4-4e26-80d9-b7501e16c1f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:07.486148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 375/737] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:47:07.487827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639933e6-d753-4f00-a667-5d8760c05ad5 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:07.488265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639933e6-d753-4f00-a667-5d8760c05ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:07.488687 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639933e6-d753-4f00-a667-5d8760c05ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:07.500603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba1bdf-fea4-4e26-80d9-b7501e16c1f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:07.502107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba1bdf-fea4-4e26-80d9-b7501e16c1f8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:07.509865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-639933e6-d753-4f00-a667-5d8760c05ad5 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:07.511366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-639933e6-d753-4f00-a667-5d8760c05ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:07.545427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:07.574106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:07.595938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 376/738] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:47:07.597097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639933e6-d753-4f00-a667-5d8760c05ad5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:07.614534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 363/739] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 07:47:07.619809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba1bdf-fea4-4e26-80d9-b7501e16c1f8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:07.744914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba1bdf-fea4-4e26-80d9-b7501e16c1f8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:07.802708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639933e6-d753-4f00-a667-5d8760c05ad5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:07.808944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3900c-d7db-4faa-9c45-195552e427b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:07.809594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3900c-d7db-4faa-9c45-195552e427b2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:07.809877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3900c-d7db-4faa-9c45-195552e427b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:07.810151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3900c-d7db-4faa-9c45-195552e427b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:07.820759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3900c-d7db-4faa-9c45-195552e427b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:07.822401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3900c-d7db-4faa-9c45-195552e427b2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-06d3900c-d7db-4faa-9c45-195552e427b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:07.838538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:07.839562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:07.840448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:07.840448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:07.840448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:07.840448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 364/740] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:07.845183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3900c-d7db-4faa-9c45-195552e427b2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:07.879032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:07.931504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e6d15-79c0-4658-87e1-5f4abcd2ef83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:07.932174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e6d15-79c0-4658-87e1-5f4abcd2ef83 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:07.932453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e6d15-79c0-4658-87e1-5f4abcd2ef83 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:07.932723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e6d15-79c0-4658-87e1-5f4abcd2ef83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:07.959660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e6d15-79c0-4658-87e1-5f4abcd2ef83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:07.959965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3900c-d7db-4faa-9c45-195552e427b2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:07.962631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e6d15-79c0-4658-87e1-5f4abcd2ef83 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-952e6d15-79c0-4658-87e1-5f4abcd2ef83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:07.980878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:07.983139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:07.984058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:07.984058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:07.984058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:07.985191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 377/741] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:07.986453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e6d15-79c0-4658-87e1-5f4abcd2ef83 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:07.999780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c25f65-9db6-413f-aff0-d63a5a8db4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:08.000838 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c25f65-9db6-413f-aff0-d63a5a8db4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.000838 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c25f65-9db6-413f-aff0-d63a5a8db4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.001134 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c25f65-9db6-413f-aff0-d63a5a8db4a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.001847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c25f65-9db6-413f-aff0-d63a5a8db4a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:08.002201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c25f65-9db6-413f-aff0-d63a5a8db4a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:08.003780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c25f65-9db6-413f-aff0-d63a5a8db4a5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:08.012796 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:08.019558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 365/742] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:08.020547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c25f65-9db6-413f-aff0-d63a5a8db4a5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:08.100755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e6d15-79c0-4658-87e1-5f4abcd2ef83 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:08.104758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:08.126799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c25f65-9db6-413f-aff0-d63a5a8db4a5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:08.135723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae088fcf-3fe6-4b76-b5a8-76d4d4ee1094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:08.136339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088fcf-3fe6-4b76-b5a8-76d4d4ee1094 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.136619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088fcf-3fe6-4b76-b5a8-76d4d4ee1094 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.136891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088fcf-3fe6-4b76-b5a8-76d4d4ee1094 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.137529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae088fcf-3fe6-4b76-b5a8-76d4d4ee1094 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:08.138928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae088fcf-3fe6-4b76-b5a8-76d4d4ee1094 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:08.160764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43a543-6899-4abf-b83b-db511a360edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:08.161361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43a543-6899-4abf-b83b-db511a360edd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.161638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43a543-6899-4abf-b83b-db511a360edd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.161910 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43a543-6899-4abf-b83b-db511a360edd None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.162542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43a543-6899-4abf-b83b-db511a360edd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:08.162878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43a543-6899-4abf-b83b-db511a360edd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:08.164221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43a543-6899-4abf-b83b-db511a360edd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:08.175030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 366/743] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:08.175801 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43a543-6899-4abf-b83b-db511a360edd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:08.228128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 378/744] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:47:07 2022] POST /identity/v3/services => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:47:08.229373 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d44747-bc1e-4598-a360-fd83f13f14d0 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.229669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d44747-bc1e-4598-a360-fd83f13f14d0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.229943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d44747-bc1e-4598-a360-fd83f13f14d0 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.231186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 379/745] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:08.232317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e98a7-95d1-4c6a-b3aa-43c3bc449afb None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.232544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e98a7-95d1-4c6a-b3aa-43c3bc449afb None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.232817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e98a7-95d1-4c6a-b3aa-43c3bc449afb None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.234248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 380/746] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:08.238822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e98a7-95d1-4c6a-b3aa-43c3bc449afb None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:08.285229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43a543-6899-4abf-b83b-db511a360edd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:08.325612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:08.330577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:08.331159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63c496-e3a2-4e8e-a425-8f3e61b6c690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:08.331706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c496-e3a2-4e8e-a425-8f3e61b6c690 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.332076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c496-e3a2-4e8e-a425-8f3e61b6c690 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.332265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c496-e3a2-4e8e-a425-8f3e61b6c690 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.340550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63c496-e3a2-4e8e-a425-8f3e61b6c690 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:08.342047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63c496-e3a2-4e8e-a425-8f3e61b6c690 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1f63c496-e3a2-4e8e-a425-8f3e61b6c690 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:08.354836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:08.355789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:08.365739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:08.365739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:08.365739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:08.365739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:08.365739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:08.365739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:08.365739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:08.365739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:08.365739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 367/747] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:08.365739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734cd92-22c5-4f68-9a64-3a03e59e889f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.365739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734cd92-22c5-4f68-9a64-3a03e59e889f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.366347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734cd92-22c5-4f68-9a64-3a03e59e889f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.366415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e98a7-95d1-4c6a-b3aa-43c3bc449afb None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:08.379722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8734cd92-22c5-4f68-9a64-3a03e59e889f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:08.380976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8734cd92-22c5-4f68-9a64-3a03e59e889f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:08.401770 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e98c5-0583-4d51-8c0d-0077ebc311ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:08.401770 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e98c5-0583-4d51-8c0d-0077ebc311ad None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.402065 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e98c5-0583-4d51-8c0d-0077ebc311ad None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.402244 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e98c5-0583-4d51-8c0d-0077ebc311ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.410709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e98c5-0583-4d51-8c0d-0077ebc311ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:08.413032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e98c5-0583-4d51-8c0d-0077ebc311ad None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-231e98c5-0583-4d51-8c0d-0077ebc311ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:08.425713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:08.426635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:08.427599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:08.427599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:08.427599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:08.427599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:08.427599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:08.427599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:08.427599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:08.427599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:08.427599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 381/748] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:08.428092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6815864-cbff-43a0-a1d4-074af1aaa256 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.428329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6815864-cbff-43a0-a1d4-074af1aaa256 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.428586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6815864-cbff-43a0-a1d4-074af1aaa256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.447500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b6815864-cbff-43a0-a1d4-074af1aaa256 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:08.448722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b6815864-cbff-43a0-a1d4-074af1aaa256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:08.464805 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 368/749] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:47:08.465660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734cd92-22c5-4f68-9a64-3a03e59e889f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:08.532084 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 382/750] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 07:47:08.533369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6815864-cbff-43a0-a1d4-074af1aaa256 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:08.571734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734cd92-22c5-4f68-9a64-3a03e59e889f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:08.603338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:08.621943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828707ce-4df4-446d-8687-46201ff8224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:08.622484 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828707ce-4df4-446d-8687-46201ff8224f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.622818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828707ce-4df4-446d-8687-46201ff8224f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.623093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828707ce-4df4-446d-8687-46201ff8224f None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.623790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828707ce-4df4-446d-8687-46201ff8224f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:08.624092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828707ce-4df4-446d-8687-46201ff8224f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:08.625527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828707ce-4df4-446d-8687-46201ff8224f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:08.644149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:08.646604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 369/751] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:08.654780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828707ce-4df4-446d-8687-46201ff8224f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:08.697496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6815864-cbff-43a0-a1d4-074af1aaa256 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:08.755653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2098afa-5c1d-4f0d-9472-37151eb91815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:08.756262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2098afa-5c1d-4f0d-9472-37151eb91815 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.756596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2098afa-5c1d-4f0d-9472-37151eb91815 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.756869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2098afa-5c1d-4f0d-9472-37151eb91815 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.757556 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2098afa-5c1d-4f0d-9472-37151eb91815 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:08.757857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2098afa-5c1d-4f0d-9472-37151eb91815 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:08.759358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2098afa-5c1d-4f0d-9472-37151eb91815 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:08.787856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 383/752] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:08.788734 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2098afa-5c1d-4f0d-9472-37151eb91815 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:08.839928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828707ce-4df4-446d-8687-46201ff8224f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:08.882769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61897d38-535c-41eb-baae-60a74f85398f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:08.883291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897d38-535c-41eb-baae-60a74f85398f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.883566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897d38-535c-41eb-baae-60a74f85398f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.884403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897d38-535c-41eb-baae-60a74f85398f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.885243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61897d38-535c-41eb-baae-60a74f85398f None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=92d4614a1449473a9b949d4192cd2a76) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:08.885550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61897d38-535c-41eb-baae-60a74f85398f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:08.886998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61897d38-535c-41eb-baae-60a74f85398f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:08.893628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 370/753] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:08 2022] GET /identity/v3/role_assignments?user.id=92d4614a1449473a9b949d4192cd2a76&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:08.893628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61897d38-535c-41eb-baae-60a74f85398f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:08.911509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2098afa-5c1d-4f0d-9472-37151eb91815 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:08.950170 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:08.959683 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:08.961950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50f29-f2e6-4492-ac99-ef2d498c6978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:08.963361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50f29-f2e6-4492-ac99-ef2d498c6978 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:08.967833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50f29-f2e6-4492-ac99-ef2d498c6978 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:08.967833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50f29-f2e6-4492-ac99-ef2d498c6978 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:08.967833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af50f29-f2e6-4492-ac99-ef2d498c6978 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=a1ae39fdd6ff4841843c9826dd88229a) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:08.967833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af50f29-f2e6-4492-ac99-ef2d498c6978 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:08.967833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af50f29-f2e6-4492-ac99-ef2d498c6978 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:08.975433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 384/754] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:08 2022] GET /identity/v3/role_assignments?user.id=a1ae39fdd6ff4841843c9826dd88229a&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=62eb3e525077429aacc0ada63300802e => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:08.976535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50f29-f2e6-4492-ac99-ef2d498c6978 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:09.077355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61897d38-535c-41eb-baae-60a74f85398f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:09.142439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eeaa4c-a5a7-47fd-b229-e33add79a950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:09.143057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eeaa4c-a5a7-47fd-b229-e33add79a950 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:09.143336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eeaa4c-a5a7-47fd-b229-e33add79a950 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:09.143608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eeaa4c-a5a7-47fd-b229-e33add79a950 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:09.165590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eeaa4c-a5a7-47fd-b229-e33add79a950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-85eeaa4c-a5a7-47fd-b229-e33add79a950 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:09.173859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:09.174905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:09.175738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 371/755] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:09.184535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eeaa4c-a5a7-47fd-b229-e33add79a950 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:09.192812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50f29-f2e6-4492-ac99-ef2d498c6978 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:09.239844 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba465d-a1b0-41c5-9d2c-055c028506d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:09.240453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba465d-a1b0-41c5-9d2c-055c028506d9 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:09.240727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba465d-a1b0-41c5-9d2c-055c028506d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:09.240997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba465d-a1b0-41c5-9d2c-055c028506d9 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:09.241644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba465d-a1b0-41c5-9d2c-055c028506d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:09.243740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba465d-a1b0-41c5-9d2c-055c028506d9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:09.304583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eeaa4c-a5a7-47fd-b229-e33add79a950 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:09.312746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:09.330999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 385/756] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:47:08 2022] POST /identity/v3/services => generated 240 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:47:09.380089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed84ee5-bbe8-4319-bfe5-a78d68e21153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:09.380719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed84ee5-bbe8-4319-bfe5-a78d68e21153 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:09.381013 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed84ee5-bbe8-4319-bfe5-a78d68e21153 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:09.381273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed84ee5-bbe8-4319-bfe5-a78d68e21153 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:09.381949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed84ee5-bbe8-4319-bfe5-a78d68e21153 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:09.382251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed84ee5-bbe8-4319-bfe5-a78d68e21153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:09.384266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed84ee5-bbe8-4319-bfe5-a78d68e21153 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:09.405174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 372/757] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:47:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:09.411095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba465d-a1b0-41c5-9d2c-055c028506d9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:09.486525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:09.539073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba465d-a1b0-41c5-9d2c-055c028506d9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:09.575943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68122a47-3102-4656-b339-eb7f6931ff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:09.576585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68122a47-3102-4656-b339-eb7f6931ff19 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:09.576868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68122a47-3102-4656-b339-eb7f6931ff19 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:09.577142 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68122a47-3102-4656-b339-eb7f6931ff19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:09.588116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68122a47-3102-4656-b339-eb7f6931ff19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:09.589562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68122a47-3102-4656-b339-eb7f6931ff19 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-68122a47-3102-4656-b339-eb7f6931ff19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:09.604738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:09.605749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:09.607009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:09.607009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:09.607009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:09.607009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 386/758] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:09.619796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed84ee5-bbe8-4319-bfe5-a78d68e21153 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:09.740961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed84ee5-bbe8-4319-bfe5-a78d68e21153 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:09.793841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64359cd-889d-4127-8464-1893956c34d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:09.794467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64359cd-889d-4127-8464-1893956c34d9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:09.794775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64359cd-889d-4127-8464-1893956c34d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:09.795053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64359cd-889d-4127-8464-1893956c34d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:09.795707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64359cd-889d-4127-8464-1893956c34d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:09.796017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64359cd-889d-4127-8464-1893956c34d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:09.797401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64359cd-889d-4127-8464-1893956c34d9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:09.814427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 373/759] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:09.820095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68122a47-3102-4656-b339-eb7f6931ff19 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:09.977977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68122a47-3102-4656-b339-eb7f6931ff19 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:09.980323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:10.013648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1d940-aa83-4ea8-baf8-eef88625db1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:10.014734 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1d940-aa83-4ea8-baf8-eef88625db1f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:10.015232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1d940-aa83-4ea8-baf8-eef88625db1f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:10.015700 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1d940-aa83-4ea8-baf8-eef88625db1f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:10.031673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1d940-aa83-4ea8-baf8-eef88625db1f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:10.033392 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1d940-aa83-4ea8-baf8-eef88625db1f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-daf1d940-aa83-4ea8-baf8-eef88625db1f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:10.047963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:10.048923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:10.049883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:10.049883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:10.049883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:10.049883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:10.049883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:10.049883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:10.049883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:10.049883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:10.049883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:10.049883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:10.052083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 387/760] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Sep 16 07:47:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:10.058637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64359cd-889d-4127-8464-1893956c34d9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:10.105172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:10.216448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64359cd-889d-4127-8464-1893956c34d9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:10.260980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215904f0-78e0-483d-9a1b-4f2752e72b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:10.261598 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215904f0-78e0-483d-9a1b-4f2752e72b57 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:10.261882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215904f0-78e0-483d-9a1b-4f2752e72b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:10.262160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215904f0-78e0-483d-9a1b-4f2752e72b57 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:10.262920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215904f0-78e0-483d-9a1b-4f2752e72b57 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:10.263234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215904f0-78e0-483d-9a1b-4f2752e72b57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:10.264714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215904f0-78e0-483d-9a1b-4f2752e72b57 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:10.288201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 374/761] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Sep 16 07:47:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:10.293871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1d940-aa83-4ea8-baf8-eef88625db1f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:10.442369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1d940-aa83-4ea8-baf8-eef88625db1f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:10.492305 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66736f17-563d-4993-91d0-06aef03b747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:10.493838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66736f17-563d-4993-91d0-06aef03b747e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:10.494321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66736f17-563d-4993-91d0-06aef03b747e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:10.494820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66736f17-563d-4993-91d0-06aef03b747e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:10.506077 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66736f17-563d-4993-91d0-06aef03b747e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:10.507777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66736f17-563d-4993-91d0-06aef03b747e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-66736f17-563d-4993-91d0-06aef03b747e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:10.521911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:10.522893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:10.523802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:10.523802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:10.523802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:10.525877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 388/762] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:10.531204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215904f0-78e0-483d-9a1b-4f2752e72b57 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:10.665080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:10.684287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215904f0-78e0-483d-9a1b-4f2752e72b57 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:10.734308 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5e192-e2ba-4826-8c38-c0b221eda9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:10.734539 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31316d-0cc6-4d49-80c2-fbff2c02fc4d None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:10.734627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31316d-0cc6-4d49-80c2-fbff2c02fc4d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:10.734891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31316d-0cc6-4d49-80c2-fbff2c02fc4d None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:10.735702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5e192-e2ba-4826-8c38-c0b221eda9ae None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:10.736175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5e192-e2ba-4826-8c38-c0b221eda9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:10.736466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 389/763] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:10.736858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5e192-e2ba-4826-8c38-c0b221eda9ae None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:10.737709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5e192-e2ba-4826-8c38-c0b221eda9ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:10.738229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5e192-e2ba-4826-8c38-c0b221eda9ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:10.744234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5e192-e2ba-4826-8c38-c0b221eda9ae None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:10.744387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1b387-75c0-4e7e-acd8-2b10361b084e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:10.744978 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1b387-75c0-4e7e-acd8-2b10361b084e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:10.744978 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1b387-75c0-4e7e-acd8-2b10361b084e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:10.772151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 375/764] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:10.778947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5e192-e2ba-4826-8c38-c0b221eda9ae None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:10.783874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d5e1b387-75c0-4e7e-acd8-2b10361b084e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:10.785264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d5e1b387-75c0-4e7e-acd8-2b10361b084e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:10.797199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:10.876126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 390/765] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:10.951316 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5e192-e2ba-4826-8c38-c0b221eda9ae None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:10.987909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72333a-94ae-4e6d-9c6f-86fc22694f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:10.988986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72333a-94ae-4e6d-9c6f-86fc22694f51 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:10.989463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72333a-94ae-4e6d-9c6f-86fc22694f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:10.989929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72333a-94ae-4e6d-9c6f-86fc22694f51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:10.997339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72333a-94ae-4e6d-9c6f-86fc22694f51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:10.999071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72333a-94ae-4e6d-9c6f-86fc22694f51 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-2e72333a-94ae-4e6d-9c6f-86fc22694f51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:11.010942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:11.011845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:11.012734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:11.012734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:11.012734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:11.012734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:11.012734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:11.012734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:11.012734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:11.012734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:11.015309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 376/766] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:11.021257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1b387-75c0-4e7e-acd8-2b10361b084e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:11.140998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1b387-75c0-4e7e-acd8-2b10361b084e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:11.184179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea27be6-94a9-4d7e-a54c-f39e1cb5ca13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:11.185177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea27be6-94a9-4d7e-a54c-f39e1cb5ca13 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:11.185656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea27be6-94a9-4d7e-a54c-f39e1cb5ca13 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:11.186122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea27be6-94a9-4d7e-a54c-f39e1cb5ca13 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:11.187403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea27be6-94a9-4d7e-a54c-f39e1cb5ca13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:11.187906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea27be6-94a9-4d7e-a54c-f39e1cb5ca13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:11.189526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea27be6-94a9-4d7e-a54c-f39e1cb5ca13 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:11.209153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 391/767] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:11.215818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72333a-94ae-4e6d-9c6f-86fc22694f51 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:11.219253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:11.283113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea27be6-94a9-4d7e-a54c-f39e1cb5ca13 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:11.292959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:11.386156 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72333a-94ae-4e6d-9c6f-86fc22694f51 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:11.462234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea27be6-94a9-4d7e-a54c-f39e1cb5ca13 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:11.476317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14331de5-6b42-4caa-8768-accd85d2b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:11.476992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14331de5-6b42-4caa-8768-accd85d2b10d None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:11.477269 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14331de5-6b42-4caa-8768-accd85d2b10d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:11.477579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14331de5-6b42-4caa-8768-accd85d2b10d None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/e32b5bf1a2524f1cbc432f38195e4c08/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:11.483946 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14331de5-6b42-4caa-8768-accd85d2b10d None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=e32b5bf1a2524f1cbc432f38195e4c08, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:11.485712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14331de5-6b42-4caa-8768-accd85d2b10d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:11.501451 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67c3b5-9317-4003-b28e-d568c56b8330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:11.509448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67c3b5-9317-4003-b28e-d568c56b8330 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:11.509448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67c3b5-9317-4003-b28e-d568c56b8330 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:11.509448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67c3b5-9317-4003-b28e-d568c56b8330 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:11.509448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67c3b5-9317-4003-b28e-d568c56b8330 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:11.509448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67c3b5-9317-4003-b28e-d568c56b8330 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-0c67c3b5-9317-4003-b28e-d568c56b8330 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:11.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:11.519400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 07:47:11.520433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:11.522826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 392/768] 10.222.0.59 () {56 vars in 1129 bytes} [Fri Sep 16 07:47:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:11.528811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67c3b5-9317-4003-b28e-d568c56b8330 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:11.584052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 377/769] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:47:11 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/e32b5bf1a2524f1cbc432f38195e4c08/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:47:11.590839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bae739-6600-449d-9672-338b1ff2c1ad None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:11.591654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bae739-6600-449d-9672-338b1ff2c1ad None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:11.592300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bae739-6600-449d-9672-338b1ff2c1ad None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:11.598819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 378/770] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:11.600625 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669446f-2c17-48c7-b795-fee0f39220a9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:11.601161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669446f-2c17-48c7-b795-fee0f39220a9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:11.601639 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669446f-2c17-48c7-b795-fee0f39220a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:11.616843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:11.643302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5669446f-2c17-48c7-b795-fee0f39220a9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:11.645003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5669446f-2c17-48c7-b795-fee0f39220a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:11.655365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67c3b5-9317-4003-b28e-d568c56b8330 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:11.697734 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4cc89-920f-41b1-bd0e-b742b8c0d9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:11.698347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4cc89-920f-41b1-bd0e-b742b8c0d9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:11.698627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4cc89-920f-41b1-bd0e-b742b8c0d9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:11.698933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4cc89-920f-41b1-bd0e-b742b8c0d9a2 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:11.699588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4cc89-920f-41b1-bd0e-b742b8c0d9a2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:11.699895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4cc89-920f-41b1-bd0e-b742b8c0d9a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:11.702174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4cc89-920f-41b1-bd0e-b742b8c0d9a2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:11.710712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 393/771] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Sep 16 07:47:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:11.710712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa9765-351b-4757-8e41-880bd88678f6 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:11.710712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa9765-351b-4757-8e41-880bd88678f6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:11.710712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa9765-351b-4757-8e41-880bd88678f6 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:11.711118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 394/772] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:11.714475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa9765-351b-4757-8e41-880bd88678f6 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:11.760124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:11.764329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 379/773] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:11.766042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cb8e7-afe4-4383-b8f4-add55d9c0012 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:11.766510 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cb8e7-afe4-4383-b8f4-add55d9c0012 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:11.766979 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cb8e7-afe4-4383-b8f4-add55d9c0012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:11.803657 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-006cb8e7-afe4-4383-b8f4-add55d9c0012 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:11.805543 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-006cb8e7-afe4-4383-b8f4-add55d9c0012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:11.873027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa9765-351b-4757-8e41-880bd88678f6 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:11.925101 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854197a3-5274-4a30-8a35-8a6ff31805ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:11.926125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854197a3-5274-4a30-8a35-8a6ff31805ea None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:11.926629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854197a3-5274-4a30-8a35-8a6ff31805ea None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:11.927140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854197a3-5274-4a30-8a35-8a6ff31805ea None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:11.928209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854197a3-5274-4a30-8a35-8a6ff31805ea None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:11.928714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854197a3-5274-4a30-8a35-8a6ff31805ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:11.930303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854197a3-5274-4a30-8a35-8a6ff31805ea None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:11.931115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 380/774] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:11.932935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:11.938812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 395/775] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Sep 16 07:47:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:11.944495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cb8e7-afe4-4383-b8f4-add55d9c0012 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:12.055525 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cb8e7-afe4-4383-b8f4-add55d9c0012 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:12.064146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854197a3-5274-4a30-8a35-8a6ff31805ea None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:12.081433 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:12.094712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c6e0f-c1b8-4030-8728-eb53bb7ffc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:12.097835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c6e0f-c1b8-4030-8728-eb53bb7ffc4c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:12.097835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c6e0f-c1b8-4030-8728-eb53bb7ffc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:12.097835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c6e0f-c1b8-4030-8728-eb53bb7ffc4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:12.097835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c6e0f-c1b8-4030-8728-eb53bb7ffc4c None admin] RBAC: Adding query filter params (interface=public, service_id=e0045387058d48d39854992ee246601b, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:12.097835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c6e0f-c1b8-4030-8728-eb53bb7ffc4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:12.098364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c6e0f-c1b8-4030-8728-eb53bb7ffc4c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:12.106929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 381/776] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Sep 16 07:47:11 2022] GET /identity/v3/endpoints?service_id=e0045387058d48d39854992ee246601b&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:12.108251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33136ef1-4797-4f60-b834-1b4f66b12ce3 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:12.108531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33136ef1-4797-4f60-b834-1b4f66b12ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:12.108806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33136ef1-4797-4f60-b834-1b4f66b12ce3 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:12.110235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 382/777] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:12.115460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33136ef1-4797-4f60-b834-1b4f66b12ce3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:12.178303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854197a3-5274-4a30-8a35-8a6ff31805ea None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:12.241340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33136ef1-4797-4f60-b834-1b4f66b12ce3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:12.280637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a532565-24e0-42c2-b911-7d214ab7a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:12.281651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532565-24e0-42c2-b911-7d214ab7a458 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:12.282197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532565-24e0-42c2-b911-7d214ab7a458 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:12.282700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532565-24e0-42c2-b911-7d214ab7a458 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:12.283427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a532565-24e0-42c2-b911-7d214ab7a458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:12.284844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a532565-24e0-42c2-b911-7d214ab7a458 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:12.294722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 383/778] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Sep 16 07:47:12 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:12.294722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1b187-f0dc-4d40-9e81-471ae03c412f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:12.294722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1b187-f0dc-4d40-9e81-471ae03c412f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:12.294722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1b187-f0dc-4d40-9e81-471ae03c412f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:12.316537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f7f1b187-f0dc-4d40-9e81-471ae03c412f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:12.318001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f7f1b187-f0dc-4d40-9e81-471ae03c412f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:12.330883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fe45f-df8c-433c-b435-c59e33af7214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:12.331593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fe45f-df8c-433c-b435-c59e33af7214 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:12.332135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fe45f-df8c-433c-b435-c59e33af7214 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:12.332246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fe45f-df8c-433c-b435-c59e33af7214 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:12.344525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5fe45f-df8c-433c-b435-c59e33af7214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-bc5fe45f-df8c-433c-b435-c59e33af7214 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:12.349717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:12.350933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 396/779] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:47:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:12.351960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fe45f-df8c-433c-b435-c59e33af7214 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:12.389362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:12.422857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 384/780] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:12.424090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1b187-f0dc-4d40-9e81-471ae03c412f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:12.449296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:12.514888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fe45f-df8c-433c-b435-c59e33af7214 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:12.562267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4b8ed-27c3-4ec9-9c61-dd74ed365a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:12.562965 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4b8ed-27c3-4ec9-9c61-dd74ed365a8d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:12.563242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4b8ed-27c3-4ec9-9c61-dd74ed365a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:12.563511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4b8ed-27c3-4ec9-9c61-dd74ed365a8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:12.574415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1b187-f0dc-4d40-9e81-471ae03c412f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:12.576739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4b8ed-27c3-4ec9-9c61-dd74ed365a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-cdb4b8ed-27c3-4ec9-9c61-dd74ed365a8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:12.581690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:12.582632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:12.583775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 397/781] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:12.589401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4b8ed-27c3-4ec9-9c61-dd74ed365a8d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:12.616691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950f4ee-57ac-4d3f-8719-7359cf358e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:12.617701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950f4ee-57ac-4d3f-8719-7359cf358e25 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:12.618235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950f4ee-57ac-4d3f-8719-7359cf358e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:12.618773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950f4ee-57ac-4d3f-8719-7359cf358e25 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:12.619695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950f4ee-57ac-4d3f-8719-7359cf358e25 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:12.620214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950f4ee-57ac-4d3f-8719-7359cf358e25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:12.621785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950f4ee-57ac-4d3f-8719-7359cf358e25 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:12.631351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 385/782] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:47:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:12.642804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950f4ee-57ac-4d3f-8719-7359cf358e25 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:12.703094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:12.727014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4b8ed-27c3-4ec9-9c61-dd74ed365a8d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:12.761894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:12.784437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950f4ee-57ac-4d3f-8719-7359cf358e25 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:12.786154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce90ea1-f4bf-47be-843e-0325ada7fe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:12.787852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce90ea1-f4bf-47be-843e-0325ada7fe62 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:12.788378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce90ea1-f4bf-47be-843e-0325ada7fe62 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:12.788865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce90ea1-f4bf-47be-843e-0325ada7fe62 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:12.789770 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce90ea1-f4bf-47be-843e-0325ada7fe62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:12.790837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce90ea1-f4bf-47be-843e-0325ada7fe62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:12.792870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce90ea1-f4bf-47be-843e-0325ada7fe62 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:12.804357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 398/783] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:47:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:12.806475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce90ea1-f4bf-47be-843e-0325ada7fe62 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:12.820808 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ee5cb-9c06-4a0b-8ff5-96af70e7ba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:12.821490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ee5cb-9c06-4a0b-8ff5-96af70e7ba9c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:12.821776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ee5cb-9c06-4a0b-8ff5-96af70e7ba9c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:12.822053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ee5cb-9c06-4a0b-8ff5-96af70e7ba9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:12.833460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ee5cb-9c06-4a0b-8ff5-96af70e7ba9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:12.834891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ee5cb-9c06-4a0b-8ff5-96af70e7ba9c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-626ee5cb-9c06-4a0b-8ff5-96af70e7ba9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:12.851748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:12.853360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:12.854303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:12.854303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:12.854303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:12.854303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 386/784] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:12.855044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ee5cb-9c06-4a0b-8ff5-96af70e7ba9c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:12.936038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce90ea1-f4bf-47be-843e-0325ada7fe62 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:12.965643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:12.974802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ee5cb-9c06-4a0b-8ff5-96af70e7ba9c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:12.978402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1604b9-27a0-4635-af4c-30eaec9ca70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:12.978402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1604b9-27a0-4635-af4c-30eaec9ca70b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:12.978402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1604b9-27a0-4635-af4c-30eaec9ca70b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:12.978402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1604b9-27a0-4635-af4c-30eaec9ca70b None admin] PATH_INFO: `/v3/services/network` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:12.978754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1604b9-27a0-4635-af4c-30eaec9ca70b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:12.979661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1604b9-27a0-4635-af4c-30eaec9ca70b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-de1604b9-27a0-4635-af4c-30eaec9ca70b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:12.988819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:12.989846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 07:47:12.990740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:12.994760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 399/785] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:12 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:12.995805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1604b9-27a0-4635-af4c-30eaec9ca70b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:13.019853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f4eaf-6fbe-4a5b-b64b-4fac2b5c66df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:13.020412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f4eaf-6fbe-4a5b-b64b-4fac2b5c66df None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:13.020676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f4eaf-6fbe-4a5b-b64b-4fac2b5c66df None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:13.020953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f4eaf-6fbe-4a5b-b64b-4fac2b5c66df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:13.032368 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f4eaf-6fbe-4a5b-b64b-4fac2b5c66df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:13.033839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f4eaf-6fbe-4a5b-b64b-4fac2b5c66df None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-e32f4eaf-6fbe-4a5b-b64b-4fac2b5c66df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:13.049050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:13.050036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:13.051558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:13.051558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:13.051558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:13.051558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 387/786] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:13.051558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f4eaf-6fbe-4a5b-b64b-4fac2b5c66df None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:13.065184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:13.139247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1604b9-27a0-4635-af4c-30eaec9ca70b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:13.159537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f4eaf-6fbe-4a5b-b64b-4fac2b5c66df None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:13.184570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb686627-944e-4e17-82a1-5fedc665782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:13.186016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb686627-944e-4e17-82a1-5fedc665782e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:13.186513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb686627-944e-4e17-82a1-5fedc665782e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:13.187317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb686627-944e-4e17-82a1-5fedc665782e None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:13.188010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb686627-944e-4e17-82a1-5fedc665782e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:13.188323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb686627-944e-4e17-82a1-5fedc665782e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:13.189691 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb686627-944e-4e17-82a1-5fedc665782e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:13.200939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c6c37-c2d2-4b7e-9ba7-7c5dde022051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:13.201819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c6c37-c2d2-4b7e-9ba7-7c5dde022051 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:13.202306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c6c37-c2d2-4b7e-9ba7-7c5dde022051 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:13.202762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c6c37-c2d2-4b7e-9ba7-7c5dde022051 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:13.203594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c6c37-c2d2-4b7e-9ba7-7c5dde022051 None admin] RBAC: Adding query filter params (name=network) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:13.203820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 400/787] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:13.204329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c6c37-c2d2-4b7e-9ba7-7c5dde022051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:13.204813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb686627-944e-4e17-82a1-5fedc665782e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:13.206247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c6c37-c2d2-4b7e-9ba7-7c5dde022051 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:13.216627 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 388/788] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:13.217588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c6c37-c2d2-4b7e-9ba7-7c5dde022051 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:13.231119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:13.322506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb686627-944e-4e17-82a1-5fedc665782e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:13.325233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:13.328230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c6c37-c2d2-4b7e-9ba7-7c5dde022051 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:13.364640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e5516-4607-47dd-9385-420a86f608f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:13.365682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e5516-4607-47dd-9385-420a86f608f6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:13.366198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e5516-4607-47dd-9385-420a86f608f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:13.366637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c7964-d460-402d-9fc5-e403919d2aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:13.366834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e5516-4607-47dd-9385-420a86f608f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:13.367513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c7964-d460-402d-9fc5-e403919d2aea None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:13.367780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e5516-4607-47dd-9385-420a86f608f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:13.368087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c7964-d460-402d-9fc5-e403919d2aea None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:13.368505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e5516-4607-47dd-9385-420a86f608f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:13.368730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c7964-d460-402d-9fc5-e403919d2aea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:13.370346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e5516-4607-47dd-9385-420a86f608f6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:13.376945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c7964-d460-402d-9fc5-e403919d2aea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:13.378518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c7964-d460-402d-9fc5-e403919d2aea None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:13.384942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 401/789] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:13.385961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e5516-4607-47dd-9385-420a86f608f6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-835c7964-d460-402d-9fc5-e403919d2aea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:13.391768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:13.392916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:13.394107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:13.394107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:13.394107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:13.394107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:13.394107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:13.394107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:13.394107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:13.394107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:13.394107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:13.394107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:13.395869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 389/790] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Sep 16 07:47:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:13.396839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c7964-d460-402d-9fc5-e403919d2aea None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:13.474590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:13.493187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e5516-4607-47dd-9385-420a86f608f6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:13.515936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c7964-d460-402d-9fc5-e403919d2aea None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:13.528730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52549a-f5e3-4aa9-812d-d8baf6ab23e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:13.529335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52549a-f5e3-4aa9-812d-d8baf6ab23e4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:13.529609 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52549a-f5e3-4aa9-812d-d8baf6ab23e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:13.529895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52549a-f5e3-4aa9-812d-d8baf6ab23e4 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:13.530514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52549a-f5e3-4aa9-812d-d8baf6ab23e4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:13.530841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52549a-f5e3-4aa9-812d-d8baf6ab23e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:13.532211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52549a-f5e3-4aa9-812d-d8baf6ab23e4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:13.538720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 402/791] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:13.538720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52549a-f5e3-4aa9-812d-d8baf6ab23e4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:13.568537 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:13.572993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfcaf1-8bc4-48c0-b177-1dfbac846580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:13.574225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfcaf1-8bc4-48c0-b177-1dfbac846580 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:13.574771 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfcaf1-8bc4-48c0-b177-1dfbac846580 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:13.575404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfcaf1-8bc4-48c0-b177-1dfbac846580 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:13.586039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfcaf1-8bc4-48c0-b177-1dfbac846580 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:13.589020 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfcaf1-8bc4-48c0-b177-1dfbac846580 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-4fbfcaf1-8bc4-48c0-b177-1dfbac846580 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:13.609671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:13.610707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:13.611675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:13.611675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:13.611675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:13.611675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:13.611675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:13.611675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:13.611675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:13.611675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:13.611675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:47:13.611675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:13.613027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 390/792] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:13.613027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfcaf1-8bc4-48c0-b177-1dfbac846580 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:13.658546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52549a-f5e3-4aa9-812d-d8baf6ab23e4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:13.701555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3abfa-5c09-4812-9c16-f163dd33d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:13.702172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3abfa-5c09-4812-9c16-f163dd33d8ed None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:13.702458 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3abfa-5c09-4812-9c16-f163dd33d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:13.703065 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3abfa-5c09-4812-9c16-f163dd33d8ed None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:13.703984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3abfa-5c09-4812-9c16-f163dd33d8ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:13.704390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3abfa-5c09-4812-9c16-f163dd33d8ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:13.705828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3abfa-5c09-4812-9c16-f163dd33d8ed None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:13.726108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 403/793] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Sep 16 07:47:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:13.730713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3abfa-5c09-4812-9c16-f163dd33d8ed None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:13.765028 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfcaf1-8bc4-48c0-b177-1dfbac846580 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:13.803085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:13.808517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a6bb1-35e4-47cf-aae8-6d64f4f10b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:13.809344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a6bb1-35e4-47cf-aae8-6d64f4f10b5a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:13.809962 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a6bb1-35e4-47cf-aae8-6d64f4f10b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:13.810438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a6bb1-35e4-47cf-aae8-6d64f4f10b5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:13.812451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a6bb1-35e4-47cf-aae8-6d64f4f10b5a None admin] RBAC: Adding query filter params (interface=public, service_id=ce99ddee5a2f447b8123a55914e1d74f, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:13.820169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a6bb1-35e4-47cf-aae8-6d64f4f10b5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:13.820169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a6bb1-35e4-47cf-aae8-6d64f4f10b5a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:13.822039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 391/794] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Sep 16 07:47:13 2022] GET /identity/v3/endpoints?service_id=ce99ddee5a2f447b8123a55914e1d74f&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:13.823278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a6bb1-35e4-47cf-aae8-6d64f4f10b5a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:13.828416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:13.845964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3abfa-5c09-4812-9c16-f163dd33d8ed None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:13.887715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f11692-1148-4f10-b96e-868e788e9b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:13.888868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f11692-1148-4f10-b96e-868e788e9b29 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:13.889270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f11692-1148-4f10-b96e-868e788e9b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:13.889624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f11692-1148-4f10-b96e-868e788e9b29 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:13.890724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f11692-1148-4f10-b96e-868e788e9b29 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:13.891113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f11692-1148-4f10-b96e-868e788e9b29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:13.892863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f11692-1148-4f10-b96e-868e788e9b29 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:13.921594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 404/795] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Sep 16 07:47:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:13.921594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f11692-1148-4f10-b96e-868e788e9b29 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:13.957678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a6bb1-35e4-47cf-aae8-6d64f4f10b5a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:14.004309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34521272-9fa7-4dd1-ac12-cf3331a937e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:14.004941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34521272-9fa7-4dd1-ac12-cf3331a937e8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.005219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34521272-9fa7-4dd1-ac12-cf3331a937e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.005486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34521272-9fa7-4dd1-ac12-cf3331a937e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.022782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34521272-9fa7-4dd1-ac12-cf3331a937e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:14.025464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34521272-9fa7-4dd1-ac12-cf3331a937e8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:14.042151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f11692-1148-4f10-b96e-868e788e9b29 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-34521272-9fa7-4dd1-ac12-cf3331a937e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:14.045493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:14.046430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:14.047365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:14.047365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:14.047365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:14.047365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 392/796] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:14.047984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34521272-9fa7-4dd1-ac12-cf3331a937e8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:14.084049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43773dd8-7d5d-4edc-ad82-17f481fe1b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:14.084787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43773dd8-7d5d-4edc-ad82-17f481fe1b07 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.085076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43773dd8-7d5d-4edc-ad82-17f481fe1b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.085351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43773dd8-7d5d-4edc-ad82-17f481fe1b07 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.085964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43773dd8-7d5d-4edc-ad82-17f481fe1b07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:14.088010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43773dd8-7d5d-4edc-ad82-17f481fe1b07 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:14.093648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 405/797] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Sep 16 07:47:13 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:14.099751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43773dd8-7d5d-4edc-ad82-17f481fe1b07 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:14.108534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:14.123086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:14.165873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34521272-9fa7-4dd1-ac12-cf3331a937e8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:14.204353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72da8a-f285-4512-98e2-457e3d769b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:14.204897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72da8a-f285-4512-98e2-457e3d769b1d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.205172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72da8a-f285-4512-98e2-457e3d769b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.205440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72da8a-f285-4512-98e2-457e3d769b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.241193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72da8a-f285-4512-98e2-457e3d769b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:14.242838 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72da8a-f285-4512-98e2-457e3d769b1d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1e72da8a-f285-4512-98e2-457e3d769b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:14.264105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:14.266278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:14.268394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:14.268394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:14.268394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:14.268394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 393/798] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:14.268394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e9534-7138-45a6-ad0d-d838c4364461 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.268394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e9534-7138-45a6-ad0d-d838c4364461 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.268394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e9534-7138-45a6-ad0d-d838c4364461 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.273996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 394/799] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:14.276167 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e9534-7138-45a6-ad0d-d838c4364461 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:14.291550 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43773dd8-7d5d-4edc-ad82-17f481fe1b07 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:14.340271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a9d2c-dd1a-4f22-816c-2174c5d4fa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:14.341276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a9d2c-dd1a-4f22-816c-2174c5d4fa41 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.341554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a9d2c-dd1a-4f22-816c-2174c5d4fa41 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.341826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a9d2c-dd1a-4f22-816c-2174c5d4fa41 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.342471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a9d2c-dd1a-4f22-816c-2174c5d4fa41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:14.342808 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a9d2c-dd1a-4f22-816c-2174c5d4fa41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:14.344717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a9d2c-dd1a-4f22-816c-2174c5d4fa41 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:14.363639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 406/800] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:14.363639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd47f96-fc8f-466a-8a78-149ebb744b69 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.363639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd47f96-fc8f-466a-8a78-149ebb744b69 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.363639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd47f96-fc8f-466a-8a78-149ebb744b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.382897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9cd47f96-fc8f-466a-8a78-149ebb744b69 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:14.384634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9cd47f96-fc8f-466a-8a78-149ebb744b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:14.397866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:14.409675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e9534-7138-45a6-ad0d-d838c4364461 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:14.446923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765dd36-b690-4c9d-9db3-d4227dac0518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:14.447822 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765dd36-b690-4c9d-9db3-d4227dac0518 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.447937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765dd36-b690-4c9d-9db3-d4227dac0518 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.448122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765dd36-b690-4c9d-9db3-d4227dac0518 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.448769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765dd36-b690-4c9d-9db3-d4227dac0518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:14.449072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765dd36-b690-4c9d-9db3-d4227dac0518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:14.450572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765dd36-b690-4c9d-9db3-d4227dac0518 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:14.464859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 395/801] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:14.465706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765dd36-b690-4c9d-9db3-d4227dac0518 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:14.466879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:14.478835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 407/802] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:14.481779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd47f96-fc8f-466a-8a78-149ebb744b69 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:14.653444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765dd36-b690-4c9d-9db3-d4227dac0518 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:14.663988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd47f96-fc8f-466a-8a78-149ebb744b69 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:14.700473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc5351-e11b-4b97-8d9e-90c8298edc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:14.700808 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66170367-3d83-4c28-95db-8a960e5925b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:14.701189 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc5351-e11b-4b97-8d9e-90c8298edc6f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.701360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66170367-3d83-4c28-95db-8a960e5925b8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.701453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc5351-e11b-4b97-8d9e-90c8298edc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.701614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66170367-3d83-4c28-95db-8a960e5925b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.701715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc5351-e11b-4b97-8d9e-90c8298edc6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.701882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66170367-3d83-4c28-95db-8a960e5925b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.709008 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc5351-e11b-4b97-8d9e-90c8298edc6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:14.710646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc5351-e11b-4b97-8d9e-90c8298edc6f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:14.711285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66170367-3d83-4c28-95db-8a960e5925b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:14.712948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66170367-3d83-4c28-95db-8a960e5925b8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-80fc5351-e11b-4b97-8d9e-90c8298edc6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:14.722038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:14.722949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:14.723866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:14.723866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:14.723866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:14.723866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:14.723866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:14.723866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:14.723866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:14.723866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:14.723866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 408/803] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:14.724858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183840a-6891-4f4a-8170-15d44b7a58a1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.724858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183840a-6891-4f4a-8170-15d44b7a58a1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.730704 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183840a-6891-4f4a-8170-15d44b7a58a1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.730704 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 409/804] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:14.731390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183840a-6891-4f4a-8170-15d44b7a58a1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-66170367-3d83-4c28-95db-8a960e5925b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:14.731777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:14.732650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:14.733535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:14.733535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:14.733535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:14.733535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:14.733535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:14.733535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:14.733535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:14.733535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:14.733535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 396/805] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:14.734430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02042-718e-4aed-b302-96ae1ccf15fc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.734672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02042-718e-4aed-b302-96ae1ccf15fc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.741854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02042-718e-4aed-b302-96ae1ccf15fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.765298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1ef02042-718e-4aed-b302-96ae1ccf15fc None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:14.767157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1ef02042-718e-4aed-b302-96ae1ccf15fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:14.784570 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:14.799040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:14.842705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183840a-6891-4f4a-8170-15d44b7a58a1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:14.850958 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 397/806] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:14.854069 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef02042-718e-4aed-b302-96ae1ccf15fc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:14.888349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc046a-7614-47af-a051-de80f5b64491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:14.889473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc046a-7614-47af-a051-de80f5b64491 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:14.889905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc046a-7614-47af-a051-de80f5b64491 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:14.890341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc046a-7614-47af-a051-de80f5b64491 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:14.891462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc046a-7614-47af-a051-de80f5b64491 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:14.891921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc046a-7614-47af-a051-de80f5b64491 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:14.894459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc046a-7614-47af-a051-de80f5b64491 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:14.910330 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 410/807] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:14.911817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc046a-7614-47af-a051-de80f5b64491 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:14.969153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef02042-718e-4aed-b302-96ae1ccf15fc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:15.008252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31458e-22b0-4dd4-83eb-e9ea4783ceb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:15.008972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31458e-22b0-4dd4-83eb-e9ea4783ceb3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:15.009347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31458e-22b0-4dd4-83eb-e9ea4783ceb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:15.009720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31458e-22b0-4dd4-83eb-e9ea4783ceb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:15.010919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31458e-22b0-4dd4-83eb-e9ea4783ceb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:15.010919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31458e-22b0-4dd4-83eb-e9ea4783ceb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:15.012613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31458e-22b0-4dd4-83eb-e9ea4783ceb3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:15.029274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 398/808] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:15.029274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31458e-22b0-4dd4-83eb-e9ea4783ceb3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:15.041343 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:15.053951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc046a-7614-47af-a051-de80f5b64491 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:15.077507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:15.101180 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095d75a-b070-4ecd-98b9-ace59b6e264e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:15.102888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095d75a-b070-4ecd-98b9-ace59b6e264e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:15.103611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095d75a-b070-4ecd-98b9-ace59b6e264e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:15.104039 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095d75a-b070-4ecd-98b9-ace59b6e264e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:15.119503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095d75a-b070-4ecd-98b9-ace59b6e264e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-2095d75a-b070-4ecd-98b9-ace59b6e264e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:15.125855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:15.126888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:15.127833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:15.130829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 411/809] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:15.131870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095d75a-b070-4ecd-98b9-ace59b6e264e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:15.145038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31458e-22b0-4dd4-83eb-e9ea4783ceb3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:15.182839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48742540-9555-4715-9285-0b853d680f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:15.183613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48742540-9555-4715-9285-0b853d680f31 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:15.183894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48742540-9555-4715-9285-0b853d680f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:15.184171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48742540-9555-4715-9285-0b853d680f31 None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/a1ae39fdd6ff4841843c9826dd88229a/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:15.186225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48742540-9555-4715-9285-0b853d680f31 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=a1ae39fdd6ff4841843c9826dd88229a, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:15.187936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48742540-9555-4715-9285-0b853d680f31 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:15.232401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095d75a-b070-4ecd-98b9-ace59b6e264e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:15.256095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 399/810] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:47:15 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/a1ae39fdd6ff4841843c9826dd88229a/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:47:15.257072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48742540-9555-4715-9285-0b853d680f31 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:15.279273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa8353-c952-45c1-83a3-2da767ebc46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:15.279878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa8353-c952-45c1-83a3-2da767ebc46d None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:15.280175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa8353-c952-45c1-83a3-2da767ebc46d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:15.280435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa8353-c952-45c1-83a3-2da767ebc46d None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/92d4614a1449473a9b949d4192cd2a76/roles/691d078c15384159b1eec2b2ceaad9ed` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:15.298832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa8353-c952-45c1-83a3-2da767ebc46d None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=92d4614a1449473a9b949d4192cd2a76, role_id=691d078c15384159b1eec2b2ceaad9ed)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:15.300657 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa8353-c952-45c1-83a3-2da767ebc46d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:15.344721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:15.383019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48742540-9555-4715-9285-0b853d680f31 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:15.384049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 412/811] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:47:15 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/92d4614a1449473a9b949d4192cd2a76/roles/691d078c15384159b1eec2b2ceaad9ed => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:47:15.385673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa8353-c952-45c1-83a3-2da767ebc46d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:15.433847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f048f77-f7db-4deb-aef8-618297707182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:15.434549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048f77-f7db-4deb-aef8-618297707182 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:15.434881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048f77-f7db-4deb-aef8-618297707182 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:15.435155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048f77-f7db-4deb-aef8-618297707182 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:15.435757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f048f77-f7db-4deb-aef8-618297707182 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:15.442710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f048f77-f7db-4deb-aef8-618297707182 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1f048f77-f7db-4deb-aef8-618297707182 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:15.450802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:15.451897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:15.453183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 400/812] 10.222.0.59 () {56 vars in 1129 bytes} [Fri Sep 16 07:47:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:15.458276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f048f77-f7db-4deb-aef8-618297707182 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:15.465415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:15.591569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa8353-c952-45c1-83a3-2da767ebc46d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:15.609873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f048f77-f7db-4deb-aef8-618297707182 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:15.652185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:15.658361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2ed7d-5e32-475c-89fa-6c44742035ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:15.659266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2ed7d-5e32-475c-89fa-6c44742035ba None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:15.663348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2ed7d-5e32-475c-89fa-6c44742035ba None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:15.663348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2ed7d-5e32-475c-89fa-6c44742035ba None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:15.663348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2ed7d-5e32-475c-89fa-6c44742035ba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:15.663348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2ed7d-5e32-475c-89fa-6c44742035ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:15.663635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88925242-7918-46e2-b927-3a1cf73082bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:15.663735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2ed7d-5e32-475c-89fa-6c44742035ba None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:15.663810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88925242-7918-46e2-b927-3a1cf73082bb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:15.663810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88925242-7918-46e2-b927-3a1cf73082bb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:15.663810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88925242-7918-46e2-b927-3a1cf73082bb None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:15.663810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88925242-7918-46e2-b927-3a1cf73082bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:15.664291 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88925242-7918-46e2-b927-3a1cf73082bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:15.666530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88925242-7918-46e2-b927-3a1cf73082bb None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:15.670633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 401/813] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Sep 16 07:47:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:15.675819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2ed7d-5e32-475c-89fa-6c44742035ba None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:15.676685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 413/814] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:47:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:15.683690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88925242-7918-46e2-b927-3a1cf73082bb None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:15.835566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:15.851548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88925242-7918-46e2-b927-3a1cf73082bb None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:15.856381 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2ed7d-5e32-475c-89fa-6c44742035ba None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:15.913370 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f01f86-320c-4728-91cb-aa23f66477c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:15.914071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f01f86-320c-4728-91cb-aa23f66477c3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:15.914446 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f01f86-320c-4728-91cb-aa23f66477c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:15.914778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f01f86-320c-4728-91cb-aa23f66477c3 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:15.914954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018ec63-deea-4460-a183-a309285ab9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:15.915419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f01f86-320c-4728-91cb-aa23f66477c3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:15.929336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018ec63-deea-4460-a183-a309285ab9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:15.929467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f01f86-320c-4728-91cb-aa23f66477c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:15.929547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018ec63-deea-4460-a183-a309285ab9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:15.929547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018ec63-deea-4460-a183-a309285ab9d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:15.929675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f01f86-320c-4728-91cb-aa23f66477c3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:15.929675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 402/815] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Sep 16 07:47:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:15.931011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f01f86-320c-4728-91cb-aa23f66477c3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:15.936807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018ec63-deea-4460-a183-a309285ab9d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:15.938516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018ec63-deea-4460-a183-a309285ab9d5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b018ec63-deea-4460-a183-a309285ab9d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:15.956167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:15.957196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:15.958551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:15.958551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:15.958551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:15.958551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 414/816] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:15.964242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018ec63-deea-4460-a183-a309285ab9d5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:16.047803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:16.113874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f01f86-320c-4728-91cb-aa23f66477c3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:16.150035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0950757-07b1-4ef3-ac2b-83f91c0abf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:16.151037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0950757-07b1-4ef3-ac2b-83f91c0abf1d None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:16.151525 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0950757-07b1-4ef3-ac2b-83f91c0abf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:16.152000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0950757-07b1-4ef3-ac2b-83f91c0abf1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:16.152926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0950757-07b1-4ef3-ac2b-83f91c0abf1d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:16.154518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0950757-07b1-4ef3-ac2b-83f91c0abf1d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:16.165890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018ec63-deea-4460-a183-a309285ab9d5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:16.209067 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc5fde-6c6c-4b4a-a5cd-d3eeff0402eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:16.210141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc5fde-6c6c-4b4a-a5cd-d3eeff0402eb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:16.210654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc5fde-6c6c-4b4a-a5cd-d3eeff0402eb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:16.211204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc5fde-6c6c-4b4a-a5cd-d3eeff0402eb None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:16.212566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc5fde-6c6c-4b4a-a5cd-d3eeff0402eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:16.213098 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc5fde-6c6c-4b4a-a5cd-d3eeff0402eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:16.214844 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc5fde-6c6c-4b4a-a5cd-d3eeff0402eb None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:16.228851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 415/817] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:16.234733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc5fde-6c6c-4b4a-a5cd-d3eeff0402eb None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:16.236766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:16.263689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 403/818] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 07:47:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:47:16.376856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc5fde-6c6c-4b4a-a5cd-d3eeff0402eb None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:16.440240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3e154-55f7-4e31-bd32-94381a587516 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:16.440803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3e154-55f7-4e31-bd32-94381a587516 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:16.440803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3e154-55f7-4e31-bd32-94381a587516 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:16.442427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 404/819] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:16.443777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9fa41-ef96-41ec-9af5-a4a078377255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:16.444428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9fa41-ef96-41ec-9af5-a4a078377255 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:16.444707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9fa41-ef96-41ec-9af5-a4a078377255 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:16.444977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9fa41-ef96-41ec-9af5-a4a078377255 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:16.448502 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5111352-865d-45c8-8f9d-942c7065d98a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:16.448789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5111352-865d-45c8-8f9d-942c7065d98a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:16.449063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5111352-865d-45c8-8f9d-942c7065d98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:16.457781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9fa41-ef96-41ec-9af5-a4a078377255 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:16.459300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9fa41-ef96-41ec-9af5-a4a078377255 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:16.473925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b5111352-865d-45c8-8f9d-942c7065d98a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:16.475181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b5111352-865d-45c8-8f9d-942c7065d98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-03f9fa41-ef96-41ec-9af5-a4a078377255 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:16.481871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:16.483017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:16.483977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:16.483977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:16.483977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:16.483977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:16.483977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:16.483977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:16.483977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:16.483977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:16.483977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:16.483977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:16.486776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 416/820] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Sep 16 07:47:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:16.491761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9fa41-ef96-41ec-9af5-a4a078377255 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:16.598855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:16.614390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 405/821] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:16.645604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9fa41-ef96-41ec-9af5-a4a078377255 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:16.679748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100586d4-d2b6-4c42-8e20-b732f3df517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:16.680860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100586d4-d2b6-4c42-8e20-b732f3df517b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:16.681133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100586d4-d2b6-4c42-8e20-b732f3df517b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:16.681405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100586d4-d2b6-4c42-8e20-b732f3df517b None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:16.682127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100586d4-d2b6-4c42-8e20-b732f3df517b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:16.682430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100586d4-d2b6-4c42-8e20-b732f3df517b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:16.683994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100586d4-d2b6-4c42-8e20-b732f3df517b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:16.702564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 417/822] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Sep 16 07:47:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:16.707738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5111352-865d-45c8-8f9d-942c7065d98a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:16.768387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:16.826261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5111352-865d-45c8-8f9d-942c7065d98a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:16.871842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d49ad-6b1b-4cdf-8a43-1589496f1d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:16.872461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d49ad-6b1b-4cdf-8a43-1589496f1d43 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:16.872738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d49ad-6b1b-4cdf-8a43-1589496f1d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:16.873006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d49ad-6b1b-4cdf-8a43-1589496f1d43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:16.901351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100586d4-d2b6-4c42-8e20-b732f3df517b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:16.986175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d49ad-6b1b-4cdf-8a43-1589496f1d43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:16.988739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d49ad-6b1b-4cdf-8a43-1589496f1d43 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:17.035654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100586d4-d2b6-4c42-8e20-b732f3df517b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:17.050520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:17.066707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:17.099911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7eff61-1e26-4448-9bea-426c9277cc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:17.100554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7eff61-1e26-4448-9bea-426c9277cc6c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:17.100826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7eff61-1e26-4448-9bea-426c9277cc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:17.101091 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7eff61-1e26-4448-9bea-426c9277cc6c None admin] PATH_INFO: `/v3/services/network` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:17.103434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7eff61-1e26-4448-9bea-426c9277cc6c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:17.104798 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7eff61-1e26-4448-9bea-426c9277cc6c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-4f7eff61-1e26-4448-9bea-426c9277cc6c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:17.115312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:17.116325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:17.117185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 418/823] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:16 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:17.125368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7eff61-1e26-4448-9bea-426c9277cc6c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:17.177587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 406/824] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:47:17.183169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d49ad-6b1b-4cdf-8a43-1589496f1d43 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:17.234767 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7eff61-1e26-4448-9bea-426c9277cc6c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:17.310947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d166a40-923e-40e4-97f3-1524b5a401fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:17.311573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d166a40-923e-40e4-97f3-1524b5a401fc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:17.312109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d166a40-923e-40e4-97f3-1524b5a401fc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:17.312406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d166a40-923e-40e4-97f3-1524b5a401fc None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:17.313047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d166a40-923e-40e4-97f3-1524b5a401fc None admin] RBAC: Adding query filter params (name=network) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:17.313352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d166a40-923e-40e4-97f3-1524b5a401fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:17.314754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d166a40-923e-40e4-97f3-1524b5a401fc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:17.326217 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 419/825] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:17.331758 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d166a40-923e-40e4-97f3-1524b5a401fc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:17.345376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d49ad-6b1b-4cdf-8a43-1589496f1d43 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:17.399889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:17.416408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a49de-4a29-47cc-a135-048c03f3f07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:17.417058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a49de-4a29-47cc-a135-048c03f3f07a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:17.417335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a49de-4a29-47cc-a135-048c03f3f07a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:17.417601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a49de-4a29-47cc-a135-048c03f3f07a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:17.440787 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a49de-4a29-47cc-a135-048c03f3f07a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:17.443030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a49de-4a29-47cc-a135-048c03f3f07a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:17.457014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:17.460747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d166a40-923e-40e4-97f3-1524b5a401fc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-078a49de-4a29-47cc-a135-048c03f3f07a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:17.476131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:17.477190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:17.478742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:17.478742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:17.478742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:17.478742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 407/826] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:17.482691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a49de-4a29-47cc-a135-048c03f3f07a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:17.509517 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450da1c-5d06-48d9-8379-560822f9c90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:17.510535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450da1c-5d06-48d9-8379-560822f9c90b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:17.510861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450da1c-5d06-48d9-8379-560822f9c90b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:17.511136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450da1c-5d06-48d9-8379-560822f9c90b None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:17.511851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450da1c-5d06-48d9-8379-560822f9c90b None admin] RBAC: Adding query filter params (type=network) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:17.512158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450da1c-5d06-48d9-8379-560822f9c90b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:17.513577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450da1c-5d06-48d9-8379-560822f9c90b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:17.527090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 420/827] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:17.532161 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450da1c-5d06-48d9-8379-560822f9c90b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:17.660815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a49de-4a29-47cc-a135-048c03f3f07a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:17.720528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450da1c-5d06-48d9-8379-560822f9c90b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:17.730843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:17.758258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bf06b-9ec5-4607-a739-88d754453561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:17.759754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bf06b-9ec5-4607-a739-88d754453561 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:17.760248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bf06b-9ec5-4607-a739-88d754453561 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:17.760753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bf06b-9ec5-4607-a739-88d754453561 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:17.761624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bf06b-9ec5-4607-a739-88d754453561 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:17.763245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bf06b-9ec5-4607-a739-88d754453561 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:17.767983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87194b4-84a1-4eeb-a588-f8ebdf4012b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:17.768761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87194b4-84a1-4eeb-a588-f8ebdf4012b3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:17.768893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87194b4-84a1-4eeb-a588-f8ebdf4012b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:17.769146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87194b4-84a1-4eeb-a588-f8ebdf4012b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:17.769803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87194b4-84a1-4eeb-a588-f8ebdf4012b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:17.770110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87194b4-84a1-4eeb-a588-f8ebdf4012b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:17.771520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87194b4-84a1-4eeb-a588-f8ebdf4012b3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:17.791252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 408/828] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:17.798752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87194b4-84a1-4eeb-a588-f8ebdf4012b3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:17.858035 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 421/829] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 07:47:17 2022] POST /identity/v3/endpoints => generated 337 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:47:17.878737 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:17.946425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87194b4-84a1-4eeb-a588-f8ebdf4012b3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:17.985360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54214881-ca8b-4fd6-b104-316a59a8391c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:17.985650 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54214881-ca8b-4fd6-b104-316a59a8391c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:17.985832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54214881-ca8b-4fd6-b104-316a59a8391c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:17.987542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 422/830] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:17.994237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68db8d3-3f66-461c-a071-08745bae739c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:17.994536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68db8d3-3f66-461c-a071-08745bae739c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:17.994823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68db8d3-3f66-461c-a071-08745bae739c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:18.041521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d68db8d3-3f66-461c-a071-08745bae739c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:18.042838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d68db8d3-3f66-461c-a071-08745bae739c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:18.153481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf92af-124d-45b4-b99c-c708bc73bcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:18.154136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf92af-124d-45b4-b99c-c708bc73bcfe None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:18.154415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf92af-124d-45b4-b99c-c708bc73bcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:18.154708 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf92af-124d-45b4-b99c-c708bc73bcfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:18.162065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf92af-124d-45b4-b99c-c708bc73bcfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:18.163553 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf92af-124d-45b4-b99c-c708bc73bcfe None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:18.172738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 423/831] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:18.174002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8e0e9-33d2-4038-9c28-cba7998f39a6 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:18.174285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8e0e9-33d2-4038-9c28-cba7998f39a6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:18.174554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8e0e9-33d2-4038-9c28-cba7998f39a6 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:18.175944 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 424/832] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:18.182945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd99f0-ad8a-4dce-afca-3d04ee246bb8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:18.182945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd99f0-ad8a-4dce-afca-3d04ee246bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:18.182945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd99f0-ad8a-4dce-afca-3d04ee246bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-14bf92af-124d-45b4-b99c-c708bc73bcfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:18.185264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:18.186136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:18.187346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:18.187346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:18.187346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:18.187346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:18.187346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:18.187346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:18.187346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:18.187346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:18.187346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 409/833] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:18.193556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf92af-124d-45b4-b99c-c708bc73bcfe None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:18.210818 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8edd99f0-ad8a-4dce-afca-3d04ee246bb8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:18.212632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8edd99f0-ad8a-4dce-afca-3d04ee246bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:18.296212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 425/834] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:18.311571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf92af-124d-45b4-b99c-c708bc73bcfe None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:18.328692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:18.348989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb19cd-f252-41c2-a988-2cccf646d79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:18.349600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb19cd-f252-41c2-a988-2cccf646d79f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:18.349892 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb19cd-f252-41c2-a988-2cccf646d79f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:18.350151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb19cd-f252-41c2-a988-2cccf646d79f None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:18.350870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb19cd-f252-41c2-a988-2cccf646d79f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:18.351176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb19cd-f252-41c2-a988-2cccf646d79f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:18.352697 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb19cd-f252-41c2-a988-2cccf646d79f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:18.364914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 410/835] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:18.371302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd99f0-ad8a-4dce-afca-3d04ee246bb8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:18.397820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:18.475376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd99f0-ad8a-4dce-afca-3d04ee246bb8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:18.502764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb19cd-f252-41c2-a988-2cccf646d79f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:18.513526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c6f13-0ccc-43ba-a38d-b6efcab77bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:18.514274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c6f13-0ccc-43ba-a38d-b6efcab77bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:18.514571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c6f13-0ccc-43ba-a38d-b6efcab77bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:18.514886 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c6f13-0ccc-43ba-a38d-b6efcab77bf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:18.515616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c6f13-0ccc-43ba-a38d-b6efcab77bf7 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=e32b5bf1a2524f1cbc432f38195e4c08) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:18.515932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c6f13-0ccc-43ba-a38d-b6efcab77bf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:18.517361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c6f13-0ccc-43ba-a38d-b6efcab77bf7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:18.524484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 426/836] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:18 2022] GET /identity/v3/role_assignments?user.id=e32b5bf1a2524f1cbc432f38195e4c08&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=62eb3e525077429aacc0ada63300802e => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:18.600266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c6f13-0ccc-43ba-a38d-b6efcab77bf7 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:18.634942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb19cd-f252-41c2-a988-2cccf646d79f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:18.674989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca76ba-e42a-4bf1-8d46-d7a094347a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:18.675649 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca76ba-e42a-4bf1-8d46-d7a094347a11 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:18.675929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca76ba-e42a-4bf1-8d46-d7a094347a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:18.676205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca76ba-e42a-4bf1-8d46-d7a094347a11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:18.689412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca76ba-e42a-4bf1-8d46-d7a094347a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-bdca76ba-e42a-4bf1-8d46-d7a094347a11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:18.694496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:18.695529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:18.696444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 411/837] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:18.703770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca76ba-e42a-4bf1-8d46-d7a094347a11 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:18.723946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:18.741738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c6f13-0ccc-43ba-a38d-b6efcab77bf7 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:18.749761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:18.787714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0616c8c-b3b7-4903-b912-8d90447ce083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:18.788484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0616c8c-b3b7-4903-b912-8d90447ce083 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:18.788823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0616c8c-b3b7-4903-b912-8d90447ce083 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:18.789294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0616c8c-b3b7-4903-b912-8d90447ce083 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:18.803693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0616c8c-b3b7-4903-b912-8d90447ce083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-c0616c8c-b3b7-4903-b912-8d90447ce083 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:18.809093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:18.810062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:18.810941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 427/838] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:47:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:18.816566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0616c8c-b3b7-4903-b912-8d90447ce083 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:18.836796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca76ba-e42a-4bf1-8d46-d7a094347a11 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:18.890228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338238e-d027-4c2d-b76a-885478e43cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:18.890228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338238e-d027-4c2d-b76a-885478e43cea None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:18.890228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338238e-d027-4c2d-b76a-885478e43cea None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:18.890228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338238e-d027-4c2d-b76a-885478e43cea None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:18.890746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338238e-d027-4c2d-b76a-885478e43cea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:18.890746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338238e-d027-4c2d-b76a-885478e43cea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:18.890746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338238e-d027-4c2d-b76a-885478e43cea None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:18.902135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 412/839] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:47:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:18.908102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338238e-d027-4c2d-b76a-885478e43cea None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:18.939807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0616c8c-b3b7-4903-b912-8d90447ce083 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:18.978149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ed04d-c80f-4be7-9b8b-ec0b977ea39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:18.978884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ed04d-c80f-4be7-9b8b-ec0b977ea39e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:18.979229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ed04d-c80f-4be7-9b8b-ec0b977ea39e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:18.979571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ed04d-c80f-4be7-9b8b-ec0b977ea39e None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:18.980357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ed04d-c80f-4be7-9b8b-ec0b977ea39e None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:18.980728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ed04d-c80f-4be7-9b8b-ec0b977ea39e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:18.982152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ed04d-c80f-4be7-9b8b-ec0b977ea39e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:18.995933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 428/840] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:47:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:18.996882 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ed04d-c80f-4be7-9b8b-ec0b977ea39e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:19.010404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:19.037187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:19.044209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338238e-d027-4c2d-b76a-885478e43cea None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:19.078530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1830eb9-3f3e-437d-8fed-6ba234e19766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:19.079206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1830eb9-3f3e-437d-8fed-6ba234e19766 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.079490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1830eb9-3f3e-437d-8fed-6ba234e19766 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.079762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1830eb9-3f3e-437d-8fed-6ba234e19766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.091921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1830eb9-3f3e-437d-8fed-6ba234e19766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:19.093389 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1830eb9-3f3e-437d-8fed-6ba234e19766 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:19.104141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ed04d-c80f-4be7-9b8b-ec0b977ea39e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d1830eb9-3f3e-437d-8fed-6ba234e19766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:19.109034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:19.109971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:19.115975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:19.115975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:19.115975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:19.115975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 413/841] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:19.115975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddf479-f070-4eec-8acc-5ab2c3e9aeaf None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.115975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddf479-f070-4eec-8acc-5ab2c3e9aeaf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.115975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddf479-f070-4eec-8acc-5ab2c3e9aeaf None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.115975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 414/842] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:19.115975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddf479-f070-4eec-8acc-5ab2c3e9aeaf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:19.150018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc69091-cbf6-470f-8295-cdbe3dc71746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:19.150600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69091-cbf6-470f-8295-cdbe3dc71746 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.150890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69091-cbf6-470f-8295-cdbe3dc71746 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.151162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69091-cbf6-470f-8295-cdbe3dc71746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.162008 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc69091-cbf6-470f-8295-cdbe3dc71746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:19.163471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc69091-cbf6-470f-8295-cdbe3dc71746 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-8cc69091-cbf6-470f-8295-cdbe3dc71746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:19.178849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:19.180009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:19.180963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:19.180963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:19.180963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:19.180963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 429/843] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:19.181192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d943c-36da-42c7-a5df-ec01420a5ced None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.181626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d943c-36da-42c7-a5df-ec01420a5ced None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.181723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d943c-36da-42c7-a5df-ec01420a5ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.200635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c51d943c-36da-42c7-a5df-ec01420a5ced None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:19.202268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c51d943c-36da-42c7-a5df-ec01420a5ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:19.229506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddf479-f070-4eec-8acc-5ab2c3e9aeaf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:19.263161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5055ac-2974-4951-b6a6-b769931cf92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:19.263765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5055ac-2974-4951-b6a6-b769931cf92c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.264050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5055ac-2974-4951-b6a6-b769931cf92c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.264378 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5055ac-2974-4951-b6a6-b769931cf92c None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.264547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:19.265023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5055ac-2974-4951-b6a6-b769931cf92c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:19.265332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5055ac-2974-4951-b6a6-b769931cf92c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:19.266721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5055ac-2974-4951-b6a6-b769931cf92c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:19.275877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 430/844] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:19.282968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d943c-36da-42c7-a5df-ec01420a5ced None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:19.292689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 415/845] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:19.293450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5055ac-2974-4951-b6a6-b769931cf92c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:19.308237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:19.391900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d943c-36da-42c7-a5df-ec01420a5ced None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:19.428125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c5ceb-2d31-4473-b2f5-df1b2c671b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:19.428766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c5ceb-2d31-4473-b2f5-df1b2c671b33 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.429042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c5ceb-2d31-4473-b2f5-df1b2c671b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.429313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c5ceb-2d31-4473-b2f5-df1b2c671b33 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.429955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c5ceb-2d31-4473-b2f5-df1b2c671b33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:19.430292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c5ceb-2d31-4473-b2f5-df1b2c671b33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:19.431761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c5ceb-2d31-4473-b2f5-df1b2c671b33 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:19.432554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5055ac-2974-4951-b6a6-b769931cf92c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:19.447036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 431/846] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:19.449842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c5ceb-2d31-4473-b2f5-df1b2c671b33 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:19.479139 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775f879-4bca-43ba-ac89-4217801e1575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:19.480035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775f879-4bca-43ba-ac89-4217801e1575 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.480324 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775f879-4bca-43ba-ac89-4217801e1575 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.481017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775f879-4bca-43ba-ac89-4217801e1575 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.490829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775f879-4bca-43ba-ac89-4217801e1575 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:19.492863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775f879-4bca-43ba-ac89-4217801e1575 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:19.506048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5775f879-4bca-43ba-ac89-4217801e1575 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:19.509980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:19.511258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:19.512214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:19.512214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:19.512214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:19.512214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:19.512214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:19.512214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:19.512214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:19.512214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:19.512214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 07:47:19.512214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:19.514765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 416/847] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:19.518043 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775f879-4bca-43ba-ac89-4217801e1575 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:19.570178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c5ceb-2d31-4473-b2f5-df1b2c671b33 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:19.604210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:19.605873 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfa131-b379-45c7-bea4-edbaf470e925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:19.606497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfa131-b379-45c7-bea4-edbaf470e925 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.606799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfa131-b379-45c7-bea4-edbaf470e925 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.607068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfa131-b379-45c7-bea4-edbaf470e925 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.614488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfa131-b379-45c7-bea4-edbaf470e925 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:19.615989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfa131-b379-45c7-bea4-edbaf470e925 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-c1cfa131-b379-45c7-bea4-edbaf470e925 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:19.628050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:19.629016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:19.630015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:19.630015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:19.630015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:19.630015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:19.630015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:19.630015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:19.630015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:19.630015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:19.630015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:19.630015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:19.631161 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 432/848] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Sep 16 07:47:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:19.631641 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfa131-b379-45c7-bea4-edbaf470e925 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:19.643151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775f879-4bca-43ba-ac89-4217801e1575 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:19.680865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322daa8e-2dc3-4ddf-b52a-90be4a608362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:19.681410 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322daa8e-2dc3-4ddf-b52a-90be4a608362 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.681693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322daa8e-2dc3-4ddf-b52a-90be4a608362 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.681966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322daa8e-2dc3-4ddf-b52a-90be4a608362 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.682744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322daa8e-2dc3-4ddf-b52a-90be4a608362 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:19.683042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322daa8e-2dc3-4ddf-b52a-90be4a608362 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:19.684506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322daa8e-2dc3-4ddf-b52a-90be4a608362 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:19.702617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 417/849] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Sep 16 07:47:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:19.707786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322daa8e-2dc3-4ddf-b52a-90be4a608362 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:19.742121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfa131-b379-45c7-bea4-edbaf470e925 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:19.756265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:19.777658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd1ce0-ce94-4778-a800-e64ac1d239b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:19.778326 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd1ce0-ce94-4778-a800-e64ac1d239b6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.778628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd1ce0-ce94-4778-a800-e64ac1d239b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.778935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd1ce0-ce94-4778-a800-e64ac1d239b6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.779519 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd1ce0-ce94-4778-a800-e64ac1d239b6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:19.780910 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd1ce0-ce94-4778-a800-e64ac1d239b6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-77cd1ce0-ce94-4778-a800-e64ac1d239b6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:19.790021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:19.791287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 07:47:19.792275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:19.796318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 433/850] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:47:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:19.797085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd1ce0-ce94-4778-a800-e64ac1d239b6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:19.820535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322daa8e-2dc3-4ddf-b52a-90be4a608362 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:19.850033 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:19.859035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf59ba-c019-4d3b-a089-45591d0d25ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:19.859653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf59ba-c019-4d3b-a089-45591d0d25ca None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.859935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf59ba-c019-4d3b-a089-45591d0d25ca None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.861095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf59ba-c019-4d3b-a089-45591d0d25ca None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.861095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf59ba-c019-4d3b-a089-45591d0d25ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:19.861095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf59ba-c019-4d3b-a089-45591d0d25ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:19.862666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf59ba-c019-4d3b-a089-45591d0d25ca None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:19.886215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 418/851] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Sep 16 07:47:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:19.887213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf59ba-c019-4d3b-a089-45591d0d25ca None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:19.912432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd1ce0-ce94-4778-a800-e64ac1d239b6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:19.950831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516ce56-abfe-40e0-b0d0-b01c5c066d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:19.951370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516ce56-abfe-40e0-b0d0-b01c5c066d56 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:19.951652 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516ce56-abfe-40e0-b0d0-b01c5c066d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:19.951929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516ce56-abfe-40e0-b0d0-b01c5c066d56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:19.992829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf59ba-c019-4d3b-a089-45591d0d25ca None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:20.000869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:20.024741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516ce56-abfe-40e0-b0d0-b01c5c066d56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:20.026292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516ce56-abfe-40e0-b0d0-b01c5c066d56 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:20.028231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d935334-251e-44c6-8884-669325632422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:20.028847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935334-251e-44c6-8884-669325632422 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:20.029130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935334-251e-44c6-8884-669325632422 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:20.029407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935334-251e-44c6-8884-669325632422 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:20.030041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d935334-251e-44c6-8884-669325632422 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:20.030347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d935334-251e-44c6-8884-669325632422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:20.031830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d935334-251e-44c6-8884-669325632422 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:20.041711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 419/852] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:47:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:20.042786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d935334-251e-44c6-8884-669325632422 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:20.094147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:20.141747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 434/853] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:19 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:47:20.141747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516ce56-abfe-40e0-b0d0-b01c5c066d56 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:20.148699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d935334-251e-44c6-8884-669325632422 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:20.182742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c07e3-7308-41ef-bfe5-f461ff45bf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:20.183359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c07e3-7308-41ef-bfe5-f461ff45bf9c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:20.183643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c07e3-7308-41ef-bfe5-f461ff45bf9c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:20.183916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c07e3-7308-41ef-bfe5-f461ff45bf9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:20.222117 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:20.245857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516ce56-abfe-40e0-b0d0-b01c5c066d56 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:20.250734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c07e3-7308-41ef-bfe5-f461ff45bf9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:20.252204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c07e3-7308-41ef-bfe5-f461ff45bf9c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:20.282826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a01d9-bdac-42ed-a220-706717bd1603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:20.283736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a01d9-bdac-42ed-a220-706717bd1603 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:20.284202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a01d9-bdac-42ed-a220-706717bd1603 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:20.284660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a01d9-bdac-42ed-a220-706717bd1603 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:20.285484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a01d9-bdac-42ed-a220-706717bd1603 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:20.285987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a01d9-bdac-42ed-a220-706717bd1603 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:20.287555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a01d9-bdac-42ed-a220-706717bd1603 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:20.292349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 435/854] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:47:20 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:20.293541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a01d9-bdac-42ed-a220-706717bd1603 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:20.308318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:20.371353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 420/855] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:20 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:47:20.377143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c07e3-7308-41ef-bfe5-f461ff45bf9c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:20.427909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a01d9-bdac-42ed-a220-706717bd1603 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:20.483360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbb893-b915-4131-9489-d39166111d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:20.483932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbb893-b915-4131-9489-d39166111d07 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:20.484222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbb893-b915-4131-9489-d39166111d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:20.484496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbb893-b915-4131-9489-d39166111d07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:20.495501 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:20.497645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbb893-b915-4131-9489-d39166111d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:20.499138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbb893-b915-4131-9489-d39166111d07 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:20.501825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c07e3-7308-41ef-bfe5-f461ff45bf9c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-00fbb893-b915-4131-9489-d39166111d07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:20.518454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:20.519522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:20.520448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:20.520448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:20.520448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:20.520448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 436/856] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:20.525500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbb893-b915-4131-9489-d39166111d07 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:20.556711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7db14e-dd6b-4c43-a11a-72b478c89668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:20.558840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db14e-dd6b-4c43-a11a-72b478c89668 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:20.559330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db14e-dd6b-4c43-a11a-72b478c89668 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:20.560047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db14e-dd6b-4c43-a11a-72b478c89668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:20.574262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7db14e-dd6b-4c43-a11a-72b478c89668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:20.576753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7db14e-dd6b-4c43-a11a-72b478c89668 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-6f7db14e-dd6b-4c43-a11a-72b478c89668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:20.596282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:20.597302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:20.598209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:20.598209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:20.598209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:20.600325 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 421/857] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:20.600441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:20.606705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7db14e-dd6b-4c43-a11a-72b478c89668 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:20.659283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbb893-b915-4131-9489-d39166111d07 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:20.704029 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937282c8-bd88-4ae0-babe-3e0433b57cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:20.704661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937282c8-bd88-4ae0-babe-3e0433b57cfe None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:20.704945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937282c8-bd88-4ae0-babe-3e0433b57cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:20.705227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937282c8-bd88-4ae0-babe-3e0433b57cfe None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:20.705898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937282c8-bd88-4ae0-babe-3e0433b57cfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:20.706208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937282c8-bd88-4ae0-babe-3e0433b57cfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:20.708494 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937282c8-bd88-4ae0-babe-3e0433b57cfe None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:20.735694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 437/858] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:20.737224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937282c8-bd88-4ae0-babe-3e0433b57cfe None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:20.748262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7db14e-dd6b-4c43-a11a-72b478c89668 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:20.785048 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f95ae4-6e47-48c2-8649-9ac60a4dbc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:20.785658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f95ae4-6e47-48c2-8649-9ac60a4dbc3f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:20.785938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f95ae4-6e47-48c2-8649-9ac60a4dbc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:20.786213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f95ae4-6e47-48c2-8649-9ac60a4dbc3f None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:20.786925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f95ae4-6e47-48c2-8649-9ac60a4dbc3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:20.787224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f95ae4-6e47-48c2-8649-9ac60a4dbc3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:20.788604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f95ae4-6e47-48c2-8649-9ac60a4dbc3f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:20.800249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:20.807067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 422/859] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:20.808055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f95ae4-6e47-48c2-8649-9ac60a4dbc3f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:20.853395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937282c8-bd88-4ae0-babe-3e0433b57cfe None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:20.871674 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:20.890392 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b729d52-810c-44d7-9cd3-a2a193421311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:20.891062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b729d52-810c-44d7-9cd3-a2a193421311 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:20.891339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b729d52-810c-44d7-9cd3-a2a193421311 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:20.891636 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b729d52-810c-44d7-9cd3-a2a193421311 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:20.898348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b729d52-810c-44d7-9cd3-a2a193421311 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:20.899874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b729d52-810c-44d7-9cd3-a2a193421311 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-4b729d52-810c-44d7-9cd3-a2a193421311 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:20.912357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:20.913283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:20.914279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:20.914279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:20.914279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:20.914279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:20.914279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:20.914279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:20.914279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:20.914279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:20.914279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 438/860] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:20.915605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c94acd-3fd7-41a1-b0f1-197033451340 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:20.915605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c94acd-3fd7-41a1-b0f1-197033451340 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:20.915731 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c94acd-3fd7-41a1-b0f1-197033451340 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:20.920922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 439/861] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:20.920922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c94acd-3fd7-41a1-b0f1-197033451340 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:20.922259 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f95ae4-6e47-48c2-8649-9ac60a4dbc3f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:20.961132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fafece-9166-4f0d-a07d-c13193963c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:20.962177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fafece-9166-4f0d-a07d-c13193963c38 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:20.962796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fafece-9166-4f0d-a07d-c13193963c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:20.963072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fafece-9166-4f0d-a07d-c13193963c38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:20.970242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fafece-9166-4f0d-a07d-c13193963c38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:20.971852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fafece-9166-4f0d-a07d-c13193963c38 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f1fafece-9166-4f0d-a07d-c13193963c38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:20.984933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:20.985900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:20.986836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:20.986836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:20.986836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:20.986836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:20.986836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:20.986836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:20.986836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:20.986836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:20.986836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 423/862] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:20.987329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9f0d8-4cb4-4420-a9ec-76e951ac0f8b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:20.987856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9f0d8-4cb4-4420-a9ec-76e951ac0f8b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:20.987856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9f0d8-4cb4-4420-a9ec-76e951ac0f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:21.006955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-72b9f0d8-4cb4-4420-a9ec-76e951ac0f8b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:21.008301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-72b9f0d8-4cb4-4420-a9ec-76e951ac0f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:21.033269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c94acd-3fd7-41a1-b0f1-197033451340 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:21.046714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:21.070276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85366e8b-36f3-45f7-84db-e8e4fdf20324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:21.071281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85366e8b-36f3-45f7-84db-e8e4fdf20324 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:21.071559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85366e8b-36f3-45f7-84db-e8e4fdf20324 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:21.071830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85366e8b-36f3-45f7-84db-e8e4fdf20324 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:21.072518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85366e8b-36f3-45f7-84db-e8e4fdf20324 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:21.072821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85366e8b-36f3-45f7-84db-e8e4fdf20324 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:21.074249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85366e8b-36f3-45f7-84db-e8e4fdf20324 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:21.079458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 424/863] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:21.080499 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9f0d8-4cb4-4420-a9ec-76e951ac0f8b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:21.092989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 440/864] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:21.098811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85366e8b-36f3-45f7-84db-e8e4fdf20324 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:21.157795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:21.219248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85366e8b-36f3-45f7-84db-e8e4fdf20324 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:21.224011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9f0d8-4cb4-4420-a9ec-76e951ac0f8b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:21.261092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f2a2d-bd55-420c-bd78-04a5899055d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:21.262015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f2a2d-bd55-420c-bd78-04a5899055d3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:21.262486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f2a2d-bd55-420c-bd78-04a5899055d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:21.263137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f2a2d-bd55-420c-bd78-04a5899055d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:21.264514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f2a2d-bd55-420c-bd78-04a5899055d3 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=a1ae39fdd6ff4841843c9826dd88229a) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:21.265123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f2a2d-bd55-420c-bd78-04a5899055d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:21.266917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f2a2d-bd55-420c-bd78-04a5899055d3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:21.268589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cd94e-f46a-4c44-b609-8a4c03a3f205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:21.269449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cd94e-f46a-4c44-b609-8a4c03a3f205 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:21.269736 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cd94e-f46a-4c44-b609-8a4c03a3f205 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:21.270006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cd94e-f46a-4c44-b609-8a4c03a3f205 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:21.270708 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cd94e-f46a-4c44-b609-8a4c03a3f205 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:21.271012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cd94e-f46a-4c44-b609-8a4c03a3f205 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:21.273234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cd94e-f46a-4c44-b609-8a4c03a3f205 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:21.275304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 441/865] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:21 2022] GET /identity/v3/role_assignments?user.id=a1ae39fdd6ff4841843c9826dd88229a&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=62eb3e525077429aacc0ada63300802e => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:21.285508 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 425/866] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:21.291656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f2a2d-bd55-420c-bd78-04a5899055d3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:21.332695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:21.374476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cd94e-f46a-4c44-b609-8a4c03a3f205 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:21.405598 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f2a2d-bd55-420c-bd78-04a5899055d3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:21.442111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84434743-50be-4e06-a6c9-2f855db5ff1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:21.442782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84434743-50be-4e06-a6c9-2f855db5ff1f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:21.443063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84434743-50be-4e06-a6c9-2f855db5ff1f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:21.443334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84434743-50be-4e06-a6c9-2f855db5ff1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:21.444060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84434743-50be-4e06-a6c9-2f855db5ff1f None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=92d4614a1449473a9b949d4192cd2a76) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:21.444368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84434743-50be-4e06-a6c9-2f855db5ff1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:21.446438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84434743-50be-4e06-a6c9-2f855db5ff1f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:21.454054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 442/867] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:21 2022] GET /identity/v3/role_assignments?user.id=92d4614a1449473a9b949d4192cd2a76&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:21.503045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cd94e-f46a-4c44-b609-8a4c03a3f205 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:21.516384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:21.545225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3bfb5-63a0-4246-9b1c-3166ed98c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:21.546287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3bfb5-63a0-4246-9b1c-3166ed98c51d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:21.546663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3bfb5-63a0-4246-9b1c-3166ed98c51d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:21.546942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3bfb5-63a0-4246-9b1c-3166ed98c51d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:21.547540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3bfb5-63a0-4246-9b1c-3166ed98c51d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:21.548960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3bfb5-63a0-4246-9b1c-3166ed98c51d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-52b3bfb5-63a0-4246-9b1c-3166ed98c51d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:21.559658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:21.561078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:21.562226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 426/868] 10.222.0.59 () {56 vars in 1132 bytes} [Fri Sep 16 07:47:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:21.567346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84434743-50be-4e06-a6c9-2f855db5ff1f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:21.678914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:21.682719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84434743-50be-4e06-a6c9-2f855db5ff1f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:21.723320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed66fe7-3b98-4caa-b415-5b629935274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:21.723948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed66fe7-3b98-4caa-b415-5b629935274d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:21.724309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed66fe7-3b98-4caa-b415-5b629935274d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:21.724588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed66fe7-3b98-4caa-b415-5b629935274d None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:21.725234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed66fe7-3b98-4caa-b415-5b629935274d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:21.725543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed66fe7-3b98-4caa-b415-5b629935274d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:21.727561 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed66fe7-3b98-4caa-b415-5b629935274d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:21.732964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 443/869] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Sep 16 07:47:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:21.740646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3bfb5-63a0-4246-9b1c-3166ed98c51d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:21.849819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3bfb5-63a0-4246-9b1c-3166ed98c51d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:21.886796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b19782-dddd-42ee-99a9-6fb048252d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:21.887807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b19782-dddd-42ee-99a9-6fb048252d19 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:21.894919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b19782-dddd-42ee-99a9-6fb048252d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:21.894919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b19782-dddd-42ee-99a9-6fb048252d19 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:21.894919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b19782-dddd-42ee-99a9-6fb048252d19 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:21.894919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b19782-dddd-42ee-99a9-6fb048252d19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:21.894919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b19782-dddd-42ee-99a9-6fb048252d19 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:21.896145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 427/870] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Sep 16 07:47:21 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:22.524946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3163a2c-05c1-4808-a6f7-f75440edb7a3 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:22.525492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3163a2c-05c1-4808-a6f7-f75440edb7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:22.525492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3163a2c-05c1-4808-a6f7-f75440edb7a3 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:22.527133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 444/871] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:22.532627 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566723f3-cb85-4bd3-8b78-bf255f2a28c8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:22.532921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566723f3-cb85-4bd3-8b78-bf255f2a28c8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:22.533195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566723f3-cb85-4bd3-8b78-bf255f2a28c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:22.551649 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-566723f3-cb85-4bd3-8b78-bf255f2a28c8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:22.552840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-566723f3-cb85-4bd3-8b78-bf255f2a28c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:22.584102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:22.626851 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 428/872] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:22.895589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3163a2c-05c1-4808-a6f7-f75440edb7a3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:23.002474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3163a2c-05c1-4808-a6f7-f75440edb7a3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:23.037119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2dd9e-e382-49ca-9054-e0ef51dc1fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:23.037724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2dd9e-e382-49ca-9054-e0ef51dc1fa1 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:23.038002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2dd9e-e382-49ca-9054-e0ef51dc1fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:23.038270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2dd9e-e382-49ca-9054-e0ef51dc1fa1 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:23.039076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2dd9e-e382-49ca-9054-e0ef51dc1fa1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:23.040980 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2dd9e-e382-49ca-9054-e0ef51dc1fa1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:23.119671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 445/873] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:47:22 2022] POST /identity/v3/services => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:47:23.540954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319d096-2c4f-4ae7-b933-2324a21e2865 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:23.540954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319d096-2c4f-4ae7-b933-2324a21e2865 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:23.540954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319d096-2c4f-4ae7-b933-2324a21e2865 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:23.541566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 429/874] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:23.546889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80085fe1-9b0a-473a-9947-f03871f29657 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:23.547735 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80085fe1-9b0a-473a-9947-f03871f29657 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:23.548209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80085fe1-9b0a-473a-9947-f03871f29657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:23.567570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-80085fe1-9b0a-473a-9947-f03871f29657 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:23.569060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-80085fe1-9b0a-473a-9947-f03871f29657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:23.588546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:23.648852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 446/875] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:23 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:23.814509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5a82a-6761-4557-adc3-e64513c517c1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:23.815053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5a82a-6761-4557-adc3-e64513c517c1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:23.815053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5a82a-6761-4557-adc3-e64513c517c1 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:23.816601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 430/876] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:23.821881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e43a4-9895-4b88-8e8a-bb3c552fea35 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:23.822170 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e43a4-9895-4b88-8e8a-bb3c552fea35 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:23.822447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e43a4-9895-4b88-8e8a-bb3c552fea35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:23.841001 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-701e43a4-9895-4b88-8e8a-bb3c552fea35 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:23.842297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-701e43a4-9895-4b88-8e8a-bb3c552fea35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:23.911908 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 447/877] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:23 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:23.933025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5a82a-6761-4557-adc3-e64513c517c1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:24.046927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5a82a-6761-4557-adc3-e64513c517c1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:24.083910 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1c50a-a907-402c-86e8-23649e1e4165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:24.084467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1c50a-a907-402c-86e8-23649e1e4165 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:24.084752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1c50a-a907-402c-86e8-23649e1e4165 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:24.085027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1c50a-a907-402c-86e8-23649e1e4165 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:24.085616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1c50a-a907-402c-86e8-23649e1e4165 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:24.087111 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1c50a-a907-402c-86e8-23649e1e4165 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-09d1c50a-a907-402c-86e8-23649e1e4165 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:24.096203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:24.097186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:24.098032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 431/878] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Sep 16 07:47:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:24.105667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e43a4-9895-4b88-8e8a-bb3c552fea35 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:24.108460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ec648-f805-4691-9cf2-cfc8e533693e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:24.108737 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ec648-f805-4691-9cf2-cfc8e533693e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:24.109036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ec648-f805-4691-9cf2-cfc8e533693e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:24.110443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 432/879] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:24.115308 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1b6e0-8036-4288-89e0-b83b4f96ec6d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:24.115595 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1b6e0-8036-4288-89e0-b83b4f96ec6d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:24.115873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1b6e0-8036-4288-89e0-b83b4f96ec6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:24.134402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b9a1b6e0-8036-4288-89e0-b83b4f96ec6d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:24.135692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b9a1b6e0-8036-4288-89e0-b83b4f96ec6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:24.163824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:24.191294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:24.210319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e43a4-9895-4b88-8e8a-bb3c552fea35 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:24.214741 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 433/880] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:24 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:24.229621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1b6e0-8036-4288-89e0-b83b4f96ec6d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:24.252471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b3f55-df57-40b2-bdd4-e9c28d03d6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:24.253376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b3f55-df57-40b2-bdd4-e9c28d03d6da None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:24.253828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b3f55-df57-40b2-bdd4-e9c28d03d6da None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:24.254286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b3f55-df57-40b2-bdd4-e9c28d03d6da None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:24.255184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b3f55-df57-40b2-bdd4-e9c28d03d6da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:24.255487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b3f55-df57-40b2-bdd4-e9c28d03d6da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:24.257125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b3f55-df57-40b2-bdd4-e9c28d03d6da None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:24.262278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 448/881] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Sep 16 07:47:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:24.267456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b3f55-df57-40b2-bdd4-e9c28d03d6da None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:24.342607 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1b6e0-8036-4288-89e0-b83b4f96ec6d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:24.383444 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b3f55-df57-40b2-bdd4-e9c28d03d6da None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:24.388120 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db726a9a-5494-4222-8ee9-0e21e0a0ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:24.389029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db726a9a-5494-4222-8ee9-0e21e0a0ac13 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:24.389506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db726a9a-5494-4222-8ee9-0e21e0a0ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:24.389973 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db726a9a-5494-4222-8ee9-0e21e0a0ac13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:24.401942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db726a9a-5494-4222-8ee9-0e21e0a0ac13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-db726a9a-5494-4222-8ee9-0e21e0a0ac13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:24.407548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:24.408478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:24.409336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:24.415230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 434/882] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Sep 16 07:47:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:24.418795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db726a9a-5494-4222-8ee9-0e21e0a0ac13 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:24.420897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32284594-d23b-4584-96ad-c929a2ffa0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:24.421511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32284594-d23b-4584-96ad-c929a2ffa0bd None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:24.421794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32284594-d23b-4584-96ad-c929a2ffa0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:24.422069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32284594-d23b-4584-96ad-c929a2ffa0bd None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:24.422722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32284594-d23b-4584-96ad-c929a2ffa0bd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:24.423040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32284594-d23b-4584-96ad-c929a2ffa0bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:24.424644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32284594-d23b-4584-96ad-c929a2ffa0bd None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:24.434702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 449/883] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Sep 16 07:47:24 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:24.452071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:24.495587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32284594-d23b-4584-96ad-c929a2ffa0bd None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:24.508958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:24.553459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db726a9a-5494-4222-8ee9-0e21e0a0ac13 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:24.604621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df99c4-7441-41c6-a5ee-cf77a883ea55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:24.605198 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df99c4-7441-41c6-a5ee-cf77a883ea55 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:24.605723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df99c4-7441-41c6-a5ee-cf77a883ea55 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:24.606007 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df99c4-7441-41c6-a5ee-cf77a883ea55 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:24.606711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df99c4-7441-41c6-a5ee-cf77a883ea55 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:24.607027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df99c4-7441-41c6-a5ee-cf77a883ea55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:24.609425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df99c4-7441-41c6-a5ee-cf77a883ea55 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:24.618389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32284594-d23b-4584-96ad-c929a2ffa0bd None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:24.620586 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 435/884] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Sep 16 07:47:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:24.626338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df99c4-7441-41c6-a5ee-cf77a883ea55 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:24.661168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb7f2b-bd65-49f1-93e3-dbb722cf490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:24.662441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb7f2b-bd65-49f1-93e3-dbb722cf490d None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:24.662441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb7f2b-bd65-49f1-93e3-dbb722cf490d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:24.662441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb7f2b-bd65-49f1-93e3-dbb722cf490d None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:24.662988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb7f2b-bd65-49f1-93e3-dbb722cf490d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:24.664400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb7f2b-bd65-49f1-93e3-dbb722cf490d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:24.729294 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:24.745965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df99c4-7441-41c6-a5ee-cf77a883ea55 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:24.779841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4d971-a5aa-4a4d-9708-bfbfd47ea790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:24.780481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4d971-a5aa-4a4d-9708-bfbfd47ea790 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:24.780815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4d971-a5aa-4a4d-9708-bfbfd47ea790 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:24.781090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4d971-a5aa-4a4d-9708-bfbfd47ea790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:24.791951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4d971-a5aa-4a4d-9708-bfbfd47ea790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:24.793424 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4d971-a5aa-4a4d-9708-bfbfd47ea790 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:24.802763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 450/885] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:47:24 2022] POST /identity/v3/services => generated 269 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-00f4d971-a5aa-4a4d-9708-bfbfd47ea790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:24.814305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:24.815853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:24.816896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:24.816896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:24.816896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:24.819364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 436/886] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:24.825962 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb7f2b-bd65-49f1-93e3-dbb722cf490d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:24.904504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:24.949183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb7f2b-bd65-49f1-93e3-dbb722cf490d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:24.990550 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebbb39-ca80-4328-8b12-bac0c3fc18ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:24.991284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebbb39-ca80-4328-8b12-bac0c3fc18ce None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:24.991571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebbb39-ca80-4328-8b12-bac0c3fc18ce None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:24.991846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebbb39-ca80-4328-8b12-bac0c3fc18ce None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:24.992521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebbb39-ca80-4328-8b12-bac0c3fc18ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:24.992829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebbb39-ca80-4328-8b12-bac0c3fc18ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:24.994196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebbb39-ca80-4328-8b12-bac0c3fc18ce None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:25.006958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 451/887] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:25.012333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4d971-a5aa-4a4d-9708-bfbfd47ea790 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:25.119306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4d971-a5aa-4a4d-9708-bfbfd47ea790 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:25.154700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a1844-1464-406c-a5ba-378b4b9f9e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:25.155284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a1844-1464-406c-a5ba-378b4b9f9e4b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:25.155568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a1844-1464-406c-a5ba-378b4b9f9e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:25.155841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a1844-1464-406c-a5ba-378b4b9f9e4b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:25.158599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:25.163878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a1844-1464-406c-a5ba-378b4b9f9e4b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:25.166065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a1844-1464-406c-a5ba-378b4b9f9e4b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-215a1844-1464-406c-a5ba-378b4b9f9e4b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:25.177870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:25.178902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:25.179895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:25.179895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:25.179895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:25.179895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:25.179895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:25.179895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:25.179895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:25.179895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:25.179895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:25.179895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:25.179895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 437/888] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Sep 16 07:47:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:25.185219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebbb39-ca80-4328-8b12-bac0c3fc18ce None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:25.296662 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebbb39-ca80-4328-8b12-bac0c3fc18ce None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:25.322186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:25.332926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1b357-d511-4717-9077-512f7147807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:25.342594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1b357-d511-4717-9077-512f7147807d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:25.342594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1b357-d511-4717-9077-512f7147807d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:25.342594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1b357-d511-4717-9077-512f7147807d None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:25.342594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1b357-d511-4717-9077-512f7147807d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:25.342594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1b357-d511-4717-9077-512f7147807d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:25.342594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1b357-d511-4717-9077-512f7147807d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:25.355207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6662b-faf8-43a3-9f38-b4b58da5a5cc None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:25.355386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 452/889] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Sep 16 07:47:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:25.355480 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6662b-faf8-43a3-9f38-b4b58da5a5cc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:25.355729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6662b-faf8-43a3-9f38-b4b58da5a5cc None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:25.357250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 438/890] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:25.361076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1b357-d511-4717-9077-512f7147807d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:25.362334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946c086-2556-4922-a6c2-5b3cb284a47e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:25.362710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946c086-2556-4922-a6c2-5b3cb284a47e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:25.363060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946c086-2556-4922-a6c2-5b3cb284a47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:25.386276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6946c086-2556-4922-a6c2-5b3cb284a47e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:25.388065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6946c086-2556-4922-a6c2-5b3cb284a47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:25.473915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1b357-d511-4717-9077-512f7147807d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:25.482813 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 439/891] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:25.514398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e2ed5-bc10-4d79-8ebd-29355dab1c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:25.515323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e2ed5-bc10-4d79-8ebd-29355dab1c3a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:25.515790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e2ed5-bc10-4d79-8ebd-29355dab1c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:25.516266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e2ed5-bc10-4d79-8ebd-29355dab1c3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:25.567716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:25.591948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e2ed5-bc10-4d79-8ebd-29355dab1c3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:25.593402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e2ed5-bc10-4d79-8ebd-29355dab1c3a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:25.700177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 453/892] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:25 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 16 07:47:25.705701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946c086-2556-4922-a6c2-5b3cb284a47e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:25.779458 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e2ed5-bc10-4d79-8ebd-29355dab1c3a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:25.810097 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946c086-2556-4922-a6c2-5b3cb284a47e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:25.837773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:25.846760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb8431-2e2a-4e95-b822-15b3ceb22360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:25.847398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb8431-2e2a-4e95-b822-15b3ceb22360 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:25.847676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb8431-2e2a-4e95-b822-15b3ceb22360 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:25.852330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb8431-2e2a-4e95-b822-15b3ceb22360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:25.858892 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb8431-2e2a-4e95-b822-15b3ceb22360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:25.860350 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb8431-2e2a-4e95-b822-15b3ceb22360 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:25.873826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-4ffb8431-2e2a-4e95-b822-15b3ceb22360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:25.875719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:25.876658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:25.877554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:25.877554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:25.877554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:25.877554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 440/893] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:25.883277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb8431-2e2a-4e95-b822-15b3ceb22360 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:25.897615 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e2ed5-bc10-4d79-8ebd-29355dab1c3a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:25.940115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4b239-5bb9-4a2a-af6e-326f146b50bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:25.940833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b239-5bb9-4a2a-af6e-326f146b50bc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:25.941114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b239-5bb9-4a2a-af6e-326f146b50bc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:25.941382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b239-5bb9-4a2a-af6e-326f146b50bc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:25.941966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4b239-5bb9-4a2a-af6e-326f146b50bc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:25.943401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4b239-5bb9-4a2a-af6e-326f146b50bc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-4af4b239-5bb9-4a2a-af6e-326f146b50bc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:25.952417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:25.954554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:25.956571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 454/894] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:47:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:25.967578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4b239-5bb9-4a2a-af6e-326f146b50bc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:26.017464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb8431-2e2a-4e95-b822-15b3ceb22360 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:26.063726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe1a01-efe0-4510-a3a7-3f10b1a605e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:26.064831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe1a01-efe0-4510-a3a7-3f10b1a605e4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:26.065292 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe1a01-efe0-4510-a3a7-3f10b1a605e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:26.065901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe1a01-efe0-4510-a3a7-3f10b1a605e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:26.067128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe1a01-efe0-4510-a3a7-3f10b1a605e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:26.067843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe1a01-efe0-4510-a3a7-3f10b1a605e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:26.070008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe1a01-efe0-4510-a3a7-3f10b1a605e4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:26.073052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4b239-5bb9-4a2a-af6e-326f146b50bc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:26.089274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 441/895] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:26.094723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe1a01-efe0-4510-a3a7-3f10b1a605e4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:26.114569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8daf9-71d3-4977-98aa-228bf2f011de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:26.115831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8daf9-71d3-4977-98aa-228bf2f011de None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:26.116345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8daf9-71d3-4977-98aa-228bf2f011de None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:26.116819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8daf9-71d3-4977-98aa-228bf2f011de None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:26.117802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8daf9-71d3-4977-98aa-228bf2f011de None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:26.118397 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8daf9-71d3-4977-98aa-228bf2f011de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:26.120031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8daf9-71d3-4977-98aa-228bf2f011de None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:26.125575 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 455/896] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:47:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:26.134157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8daf9-71d3-4977-98aa-228bf2f011de None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:26.137185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:26.149828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:26.203377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe1a01-efe0-4510-a3a7-3f10b1a605e4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:26.241077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a987ae-4440-42a2-864f-55369544fc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:26.241719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a987ae-4440-42a2-864f-55369544fc6e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:26.241999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a987ae-4440-42a2-864f-55369544fc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:26.242291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a987ae-4440-42a2-864f-55369544fc6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:26.249698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a987ae-4440-42a2-864f-55369544fc6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:26.251326 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a987ae-4440-42a2-864f-55369544fc6e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:26.261977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8daf9-71d3-4977-98aa-228bf2f011de None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-e0a987ae-4440-42a2-864f-55369544fc6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:26.263965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:26.264861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:26.265790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:26.265790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:26.265790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:26.265790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:26.265790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:26.265790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:26.265790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:26.265790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:26.265790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 442/897] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:26.270567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a987ae-4440-42a2-864f-55369544fc6e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:26.309414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381f820-572c-4768-aafe-c6591740aaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:26.310428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381f820-572c-4768-aafe-c6591740aaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:26.310931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381f820-572c-4768-aafe-c6591740aaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:26.311452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381f820-572c-4768-aafe-c6591740aaa3 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:26.312546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381f820-572c-4768-aafe-c6591740aaa3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:26.313093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381f820-572c-4768-aafe-c6591740aaa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:26.315276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381f820-572c-4768-aafe-c6591740aaa3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:26.326247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 456/898] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:47:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:26.331866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381f820-572c-4768-aafe-c6591740aaa3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:26.383235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:26.388349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a987ae-4440-42a2-864f-55369544fc6e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:26.411909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:26.426706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377d02a-b08c-4643-98ba-69e922fc6935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:26.427690 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377d02a-b08c-4643-98ba-69e922fc6935 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:26.428237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377d02a-b08c-4643-98ba-69e922fc6935 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:26.428783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377d02a-b08c-4643-98ba-69e922fc6935 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:26.429680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377d02a-b08c-4643-98ba-69e922fc6935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:26.430216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377d02a-b08c-4643-98ba-69e922fc6935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:26.431850 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377d02a-b08c-4643-98ba-69e922fc6935 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:26.440321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381f820-572c-4768-aafe-c6591740aaa3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:26.445249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 443/899] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:26.454713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377d02a-b08c-4643-98ba-69e922fc6935 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:26.481489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8d5fd-834e-4a61-b717-db1225ee73b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:26.482297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8d5fd-834e-4a61-b717-db1225ee73b4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:26.482669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8d5fd-834e-4a61-b717-db1225ee73b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:26.483119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8d5fd-834e-4a61-b717-db1225ee73b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:26.484002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8d5fd-834e-4a61-b717-db1225ee73b4 None admin] RBAC: Adding query filter params (interface=public, service_id=5d69fd7065da4e33b1b051899b5e50da, region_id=RegionOne) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:26.484439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8d5fd-834e-4a61-b717-db1225ee73b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:26.486056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8d5fd-834e-4a61-b717-db1225ee73b4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:26.494891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 457/900] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Sep 16 07:47:26 2022] GET /identity/v3/endpoints?service_id=5d69fd7065da4e33b1b051899b5e50da&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:26.500919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8d5fd-834e-4a61-b717-db1225ee73b4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:26.569344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377d02a-b08c-4643-98ba-69e922fc6935 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:26.622437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd46fa1-dfb3-4012-b71d-2e510c75fdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:26.623152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd46fa1-dfb3-4012-b71d-2e510c75fdda None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:26.623442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd46fa1-dfb3-4012-b71d-2e510c75fdda None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:26.623719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd46fa1-dfb3-4012-b71d-2e510c75fdda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:26.624707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd46fa1-dfb3-4012-b71d-2e510c75fdda None admin] RBAC: Adding query filter params (role.id=cd21a152b61c4235ad578e9b7a40d857, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=92d4614a1449473a9b949d4192cd2a76) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:26.625025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd46fa1-dfb3-4012-b71d-2e510c75fdda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:26.627196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd46fa1-dfb3-4012-b71d-2e510c75fdda None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:26.629092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8d5fd-834e-4a61-b717-db1225ee73b4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:26.634445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 444/901] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:26 2022] GET /identity/v3/role_assignments?user.id=92d4614a1449473a9b949d4192cd2a76&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=cd21a152b61c4235ad578e9b7a40d857 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:26.675178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f049e8-7cba-4bad-8795-6b305b7364f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:26.676219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f049e8-7cba-4bad-8795-6b305b7364f8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:26.676660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f049e8-7cba-4bad-8795-6b305b7364f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:26.677088 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f049e8-7cba-4bad-8795-6b305b7364f8 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:26.677942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f049e8-7cba-4bad-8795-6b305b7364f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:26.679606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f049e8-7cba-4bad-8795-6b305b7364f8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:26.687421 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 458/902] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Sep 16 07:47:26 2022] GET /identity/v3/services => generated 1268 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 07:47:26.805672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b80d2b-7c83-4ee0-8ee9-e6237cfb5093 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:26.806318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b80d2b-7c83-4ee0-8ee9-e6237cfb5093 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:26.806822 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b80d2b-7c83-4ee0-8ee9-e6237cfb5093 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:26.808774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 445/903] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:26.815494 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188cca-3e5c-41c5-947e-8fdae3190642 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:26.816436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188cca-3e5c-41c5-947e-8fdae3190642 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:26.816925 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188cca-3e5c-41c5-947e-8fdae3190642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:26.824719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:26.840339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-29188cca-3e5c-41c5-947e-8fdae3190642 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:26.841664 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-29188cca-3e5c-41c5-947e-8fdae3190642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:26.918233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 459/904] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:27.106234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19d84c-5678-483a-ac05-0cca8effbf24 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:27.106777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19d84c-5678-483a-ac05-0cca8effbf24 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:27.106777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19d84c-5678-483a-ac05-0cca8effbf24 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:27.108273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 446/905] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:27.113417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a7082-8517-47bf-857b-d8f036523df7 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:27.113698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a7082-8517-47bf-857b-d8f036523df7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:27.113971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a7082-8517-47bf-857b-d8f036523df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:27.132957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-659a7082-8517-47bf-857b-d8f036523df7 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:27.134138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-659a7082-8517-47bf-857b-d8f036523df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:27.193593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19d84c-5678-483a-ac05-0cca8effbf24 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:27.240402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:27.252756 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 460/906] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:27.393441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19d84c-5678-483a-ac05-0cca8effbf24 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:27.457144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fa794-b54a-4041-bc8e-8bafa0b5245d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:27.458133 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fa794-b54a-4041-bc8e-8bafa0b5245d None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:27.458966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fa794-b54a-4041-bc8e-8bafa0b5245d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:27.459747 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fa794-b54a-4041-bc8e-8bafa0b5245d None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:27.460884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fa794-b54a-4041-bc8e-8bafa0b5245d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:27.463693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fa794-b54a-4041-bc8e-8bafa0b5245d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:27.525591 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a7082-8517-47bf-857b-d8f036523df7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:27.559124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 447/907] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:47:27 2022] POST /identity/v3/services => generated 241 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:47:27.639082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:27.708255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a7082-8517-47bf-857b-d8f036523df7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:27.751497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c10dab-3e94-46b1-bbb8-df63bee06ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:27.752141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c10dab-3e94-46b1-bbb8-df63bee06ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:27.752418 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c10dab-3e94-46b1-bbb8-df63bee06ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:27.752691 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c10dab-3e94-46b1-bbb8-df63bee06ff8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:27.753280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c10dab-3e94-46b1-bbb8-df63bee06ff8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:27.754671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c10dab-3e94-46b1-bbb8-df63bee06ff8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-88c10dab-3e94-46b1-bbb8-df63bee06ff8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:27.764930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:27.765976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:27.766905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 461/908] 10.222.0.59 () {56 vars in 1132 bytes} [Fri Sep 16 07:47:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:27.772473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fa794-b54a-4041-bc8e-8bafa0b5245d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:27.863689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:27.882659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fa794-b54a-4041-bc8e-8bafa0b5245d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:27.920485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d8111-d107-4c43-b306-a8a805b23d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:27.921105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8111-d107-4c43-b306-a8a805b23d92 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:27.921392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8111-d107-4c43-b306-a8a805b23d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:27.921665 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8111-d107-4c43-b306-a8a805b23d92 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:27.922386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d8111-d107-4c43-b306-a8a805b23d92 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:27.922712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d8111-d107-4c43-b306-a8a805b23d92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:27.924107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d8111-d107-4c43-b306-a8a805b23d92 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:27.929406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 448/909] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Sep 16 07:47:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:27.935204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c10dab-3e94-46b1-bbb8-df63bee06ff8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:28.038665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c10dab-3e94-46b1-bbb8-df63bee06ff8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:28.062349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:28.073660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e1bdf-9e28-4310-99c1-db32006c555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:28.074269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e1bdf-9e28-4310-99c1-db32006c555a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:28.074547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e1bdf-9e28-4310-99c1-db32006c555a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:28.074849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e1bdf-9e28-4310-99c1-db32006c555a None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:28.079251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e1bdf-9e28-4310-99c1-db32006c555a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:28.079251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e1bdf-9e28-4310-99c1-db32006c555a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:28.079251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e1bdf-9e28-4310-99c1-db32006c555a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:28.082163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 462/910] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Sep 16 07:47:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:28.088483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d8111-d107-4c43-b306-a8a805b23d92 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:28.190110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d8111-d107-4c43-b306-a8a805b23d92 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:28.224723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87dd42-67c2-47e8-a577-42a49aa92fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:28.225620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87dd42-67c2-47e8-a577-42a49aa92fda None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:28.226096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87dd42-67c2-47e8-a577-42a49aa92fda None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:28.226561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87dd42-67c2-47e8-a577-42a49aa92fda None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:28.228062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87dd42-67c2-47e8-a577-42a49aa92fda None admin] RBAC: Adding query filter params (interface=public, service_id=7160de35cb9f478d9f399097daf2b9fa, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:28.228543 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87dd42-67c2-47e8-a577-42a49aa92fda None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:28.230076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87dd42-67c2-47e8-a577-42a49aa92fda None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:28.235586 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 449/911] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Sep 16 07:47:28 2022] GET /identity/v3/endpoints?service_id=7160de35cb9f478d9f399097daf2b9fa&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:28.241234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e1bdf-9e28-4310-99c1-db32006c555a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:28.346106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e1bdf-9e28-4310-99c1-db32006c555a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:28.381032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e3e80-4f4b-488e-ab06-74f31b50fb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:28.381943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3e80-4f4b-488e-ab06-74f31b50fb9e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:28.382418 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3e80-4f4b-488e-ab06-74f31b50fb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:28.383453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3e80-4f4b-488e-ab06-74f31b50fb9e None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:28.384251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e3e80-4f4b-488e-ab06-74f31b50fb9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:28.385779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e3e80-4f4b-488e-ab06-74f31b50fb9e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:28.392493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 463/912] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Sep 16 07:47:28 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 07:47:28.952242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d74ae2-cc67-41aa-a419-31e36deb8d23 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:28.952611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d74ae2-cc67-41aa-a419-31e36deb8d23 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:28.952829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d74ae2-cc67-41aa-a419-31e36deb8d23 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:28.954382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 450/913] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:28.959815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1055d-2380-4083-8814-1e7aca588804 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:28.960279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1055d-2380-4083-8814-1e7aca588804 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:28.960356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1055d-2380-4083-8814-1e7aca588804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:28.969949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd9ee7-109b-4f8a-9725-33d2916da73c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:28.970301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd9ee7-109b-4f8a-9725-33d2916da73c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:28.970513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd9ee7-109b-4f8a-9725-33d2916da73c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:28.972092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 451/914] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:28.980394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0569c06-7324-459e-bb17-72d7044e313d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:28.980792 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0569c06-7324-459e-bb17-72d7044e313d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:28.981187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0569c06-7324-459e-bb17-72d7044e313d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:28.982814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4ae1055d-2380-4083-8814-1e7aca588804 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:28.984278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4ae1055d-2380-4083-8814-1e7aca588804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:28.999454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d0569c06-7324-459e-bb17-72d7044e313d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:29.000702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d0569c06-7324-459e-bb17-72d7044e313d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:29.011595 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:29.041507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:29.069961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 464/915] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:29.075592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 452/916] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:29.337835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1055d-2380-4083-8814-1e7aca588804 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:29.356453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0569c06-7324-459e-bb17-72d7044e313d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:29.461256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0569c06-7324-459e-bb17-72d7044e313d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:29.463402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1055d-2380-4083-8814-1e7aca588804 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:29.499166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31728dab-56b1-4b76-b672-8b83d766394d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:29.500151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728dab-56b1-4b76-b672-8b83d766394d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:29.500719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728dab-56b1-4b76-b672-8b83d766394d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:29.501317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728dab-56b1-4b76-b672-8b83d766394d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:29.502175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31728dab-56b1-4b76-b672-8b83d766394d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:29.503804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31728dab-56b1-4b76-b672-8b83d766394d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:29.506239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13722c-d704-4484-9087-2ac0f820d00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:29.507270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13722c-d704-4484-9087-2ac0f820d00b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:29.507559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13722c-d704-4484-9087-2ac0f820d00b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:29.508461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13722c-d704-4484-9087-2ac0f820d00b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-31728dab-56b1-4b76-b672-8b83d766394d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:29.515076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:29.516108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 07:47:29.516967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:29.519108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13722c-d704-4484-9087-2ac0f820d00b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-1c13722c-d704-4484-9087-2ac0f820d00b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:29.523891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:29.524862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:29.525694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:29.526331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 453/917] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:47:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:29.526331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31728dab-56b1-4b76-b672-8b83d766394d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:29.528135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 465/918] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Sep 16 07:47:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:29.535894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:29.536999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13722c-d704-4484-9087-2ac0f820d00b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:29.594273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:29.653242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31728dab-56b1-4b76-b672-8b83d766394d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:29.671384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13722c-d704-4484-9087-2ac0f820d00b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:29.696693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baaf41f-6bbe-4fa9-8b04-a92780f31250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:29.697684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaf41f-6bbe-4fa9-8b04-a92780f31250 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:29.698110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaf41f-6bbe-4fa9-8b04-a92780f31250 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:29.698900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaf41f-6bbe-4fa9-8b04-a92780f31250 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:29.699961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baaf41f-6bbe-4fa9-8b04-a92780f31250 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:29.700408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baaf41f-6bbe-4fa9-8b04-a92780f31250 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:29.702409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baaf41f-6bbe-4fa9-8b04-a92780f31250 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:29.708637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 454/919] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:47:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:29.714115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baaf41f-6bbe-4fa9-8b04-a92780f31250 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:29.714552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763189c7-80ff-461b-924f-c81a4357a443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:29.715536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763189c7-80ff-461b-924f-c81a4357a443 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:29.716007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763189c7-80ff-461b-924f-c81a4357a443 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:29.716525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763189c7-80ff-461b-924f-c81a4357a443 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:29.717403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763189c7-80ff-461b-924f-c81a4357a443 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:29.717898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763189c7-80ff-461b-924f-c81a4357a443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:29.719484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763189c7-80ff-461b-924f-c81a4357a443 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:29.733319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 466/920] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Sep 16 07:47:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:29.734460 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763189c7-80ff-461b-924f-c81a4357a443 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:29.790450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:29.831912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baaf41f-6bbe-4fa9-8b04-a92780f31250 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:29.870974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58431cf6-b937-48a8-9fe9-2241301f417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:29.872524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58431cf6-b937-48a8-9fe9-2241301f417c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:29.872953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58431cf6-b937-48a8-9fe9-2241301f417c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:29.873397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58431cf6-b937-48a8-9fe9-2241301f417c None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:29.874235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58431cf6-b937-48a8-9fe9-2241301f417c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:29.874729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763189c7-80ff-461b-924f-c81a4357a443 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:29.874914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58431cf6-b937-48a8-9fe9-2241301f417c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:29.876116 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58431cf6-b937-48a8-9fe9-2241301f417c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:29.877043 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:29.890884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 455/921] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Sep 16 07:47:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:29.893746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58431cf6-b937-48a8-9fe9-2241301f417c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:29.925622 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3969f3-2c1d-42e6-a2a6-8d9125e087b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:29.926268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3969f3-2c1d-42e6-a2a6-8d9125e087b8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:29.926711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3969f3-2c1d-42e6-a2a6-8d9125e087b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:29.926997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3969f3-2c1d-42e6-a2a6-8d9125e087b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:29.939417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3969f3-2c1d-42e6-a2a6-8d9125e087b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:29.940892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3969f3-2c1d-42e6-a2a6-8d9125e087b8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-3c3969f3-2c1d-42e6-a2a6-8d9125e087b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:29.955631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:29.956694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:29.957629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:29.957629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:29.957629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:29.957629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 467/922] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:29.958549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7a48-eec6-4f94-9430-41a332ab3c3c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:29.958549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7a48-eec6-4f94-9430-41a332ab3c3c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:29.958747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7a48-eec6-4f94-9430-41a332ab3c3c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:29.959929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 468/923] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:29.963681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e7a48-eec6-4f94-9430-41a332ab3c3c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:30.020993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58431cf6-b937-48a8-9fe9-2241301f417c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:30.054022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:30.060034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a8b10-754d-43af-903e-992f43e669ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:30.060751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a8b10-754d-43af-903e-992f43e669ec None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:30.061023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a8b10-754d-43af-903e-992f43e669ec None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:30.061299 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a8b10-754d-43af-903e-992f43e669ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:30.062220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a8b10-754d-43af-903e-992f43e669ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:30.063825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a8b10-754d-43af-903e-992f43e669ec None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:30.070400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e7a48-eec6-4f94-9430-41a332ab3c3c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:30.105234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb73bc-014a-4b24-8df3-e3295b975309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:30.105854 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb73bc-014a-4b24-8df3-e3295b975309 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:30.106175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb73bc-014a-4b24-8df3-e3295b975309 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:30.106447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb73bc-014a-4b24-8df3-e3295b975309 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:30.107126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb73bc-014a-4b24-8df3-e3295b975309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:30.107433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb73bc-014a-4b24-8df3-e3295b975309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:30.108804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb73bc-014a-4b24-8df3-e3295b975309 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:30.120730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 456/924] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 07:47:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:47:30.126305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30c75a-ddce-4bef-93b0-878ea7564dbf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:30.126305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30c75a-ddce-4bef-93b0-878ea7564dbf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:30.126305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30c75a-ddce-4bef-93b0-878ea7564dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:30.127026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 469/925] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:30.127868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb73bc-014a-4b24-8df3-e3295b975309 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:30.156655 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5a30c75a-ddce-4bef-93b0-878ea7564dbf None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:30.158028 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5a30c75a-ddce-4bef-93b0-878ea7564dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:30.160041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:30.267368 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 457/926] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:30.277271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb73bc-014a-4b24-8df3-e3295b975309 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:30.318452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f0544-97fc-42ba-8da1-d58fabdab980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:30.319126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f0544-97fc-42ba-8da1-d58fabdab980 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:30.319409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f0544-97fc-42ba-8da1-d58fabdab980 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:30.319684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f0544-97fc-42ba-8da1-d58fabdab980 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:30.326896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f0544-97fc-42ba-8da1-d58fabdab980 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:30.328332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f0544-97fc-42ba-8da1-d58fabdab980 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-952f0544-97fc-42ba-8da1-d58fabdab980 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:30.340730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:30.341752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:30.342824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:30.342824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:30.342824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:30.342824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:30.342824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:30.342824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:30.342824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:30.342824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:30.342824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:30.342824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:30.342824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 470/927] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Sep 16 07:47:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:30.347693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30c75a-ddce-4bef-93b0-878ea7564dbf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:30.454131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30c75a-ddce-4bef-93b0-878ea7564dbf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:30.471110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:30.492691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62790ca-22f1-4895-9ca4-55eec916405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:30.493301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62790ca-22f1-4895-9ca4-55eec916405f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:30.493578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62790ca-22f1-4895-9ca4-55eec916405f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:30.493848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62790ca-22f1-4895-9ca4-55eec916405f None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:30.494581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62790ca-22f1-4895-9ca4-55eec916405f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:30.494929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62790ca-22f1-4895-9ca4-55eec916405f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:30.496521 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62790ca-22f1-4895-9ca4-55eec916405f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:30.514840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 458/928] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Sep 16 07:47:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:30.520783 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f0544-97fc-42ba-8da1-d58fabdab980 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:30.560020 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3d926-d532-4315-848d-14c6f27d5fb8 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:30.560386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3d926-d532-4315-848d-14c6f27d5fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:30.560578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3d926-d532-4315-848d-14c6f27d5fb8 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:30.562479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 459/929] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:30.567905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9b902-2cca-412c-9d04-adac3e5115f2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:30.568303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9b902-2cca-412c-9d04-adac3e5115f2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:30.568820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9b902-2cca-412c-9d04-adac3e5115f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:30.588795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-10c9b902-2cca-412c-9d04-adac3e5115f2 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:30.589990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-10c9b902-2cca-412c-9d04-adac3e5115f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:30.605300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:30.631138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f0544-97fc-42ba-8da1-d58fabdab980 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:30.663559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 460/930] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:30.664222 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9b902-2cca-412c-9d04-adac3e5115f2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:30.675122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487b4d1-bbf5-466d-9d80-c0275fbe887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:30.676487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487b4d1-bbf5-466d-9d80-c0275fbe887a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:30.677267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487b4d1-bbf5-466d-9d80-c0275fbe887a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:30.677760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487b4d1-bbf5-466d-9d80-c0275fbe887a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:30.692265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487b4d1-bbf5-466d-9d80-c0275fbe887a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:30.693710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487b4d1-bbf5-466d-9d80-c0275fbe887a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-3487b4d1-bbf5-466d-9d80-c0275fbe887a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:30.714396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:30.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:30.716648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:30.716648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:30.716648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:30.728427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 471/931] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:30.729622 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487b4d1-bbf5-466d-9d80-c0275fbe887a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:30.785109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9b902-2cca-412c-9d04-adac3e5115f2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:30.787117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:30.836502 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397516b3-8f08-4ff5-807f-8d6b03db3710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:30.837556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397516b3-8f08-4ff5-807f-8d6b03db3710 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:30.837953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397516b3-8f08-4ff5-807f-8d6b03db3710 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:30.838791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397516b3-8f08-4ff5-807f-8d6b03db3710 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:30.839762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397516b3-8f08-4ff5-807f-8d6b03db3710 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:30.841537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397516b3-8f08-4ff5-807f-8d6b03db3710 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:30.845001 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487b4d1-bbf5-466d-9d80-c0275fbe887a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-397516b3-8f08-4ff5-807f-8d6b03db3710 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:30.853406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:30.854476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:30.855474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 461/932] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Sep 16 07:47:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:30.860356 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397516b3-8f08-4ff5-807f-8d6b03db3710 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:30.890747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:30.902357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f4244-4889-4491-998a-65c5a6be0ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:30.903441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f4244-4889-4491-998a-65c5a6be0ced None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:30.903725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f4244-4889-4491-998a-65c5a6be0ced None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:30.904018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f4244-4889-4491-998a-65c5a6be0ced None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:30.904669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f4244-4889-4491-998a-65c5a6be0ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:30.904981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f4244-4889-4491-998a-65c5a6be0ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:30.906434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f4244-4889-4491-998a-65c5a6be0ced None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:30.923488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 472/933] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:30.924824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f4244-4889-4491-998a-65c5a6be0ced None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:30.981174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397516b3-8f08-4ff5-807f-8d6b03db3710 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:31.015094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d83e48-0067-49df-b5c3-49879178aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:31.015698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d83e48-0067-49df-b5c3-49879178aeaa None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:31.015970 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d83e48-0067-49df-b5c3-49879178aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:31.016236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d83e48-0067-49df-b5c3-49879178aeaa None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:31.016857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d83e48-0067-49df-b5c3-49879178aeaa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:31.017155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d83e48-0067-49df-b5c3-49879178aeaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:31.018521 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d83e48-0067-49df-b5c3-49879178aeaa None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:31.023056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 462/934] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Sep 16 07:47:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:31.023858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d83e48-0067-49df-b5c3-49879178aeaa None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:31.036151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f4244-4889-4491-998a-65c5a6be0ced None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:31.055382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:31.075896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bc46e-b3b0-439f-8412-1c53cec20511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:31.080718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc46e-b3b0-439f-8412-1c53cec20511 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:31.080718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc46e-b3b0-439f-8412-1c53cec20511 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:31.080718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc46e-b3b0-439f-8412-1c53cec20511 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:31.083941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bc46e-b3b0-439f-8412-1c53cec20511 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:31.085362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bc46e-b3b0-439f-8412-1c53cec20511 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-f03bc46e-b3b0-439f-8412-1c53cec20511 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:31.097520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:31.098521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:31.105007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:31.105007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:31.105007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:31.105007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:31.105007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:31.105007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:31.105007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:31.105007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:31.105007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 473/935] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:31.105007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bc46e-b3b0-439f-8412-1c53cec20511 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:31.132074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d83e48-0067-49df-b5c3-49879178aeaa None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:31.153836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:31.166854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7ebc9-8f32-4b21-ac0e-d1eb41a0b81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:31.167487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7ebc9-8f32-4b21-ac0e-d1eb41a0b81a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:31.167769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7ebc9-8f32-4b21-ac0e-d1eb41a0b81a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:31.168043 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7ebc9-8f32-4b21-ac0e-d1eb41a0b81a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:31.168708 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7ebc9-8f32-4b21-ac0e-d1eb41a0b81a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:31.170085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7ebc9-8f32-4b21-ac0e-d1eb41a0b81a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-77b7ebc9-8f32-4b21-ac0e-d1eb41a0b81a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:31.181317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:31.182272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:31.183251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 463/936] 10.222.0.59 () {56 vars in 1132 bytes} [Fri Sep 16 07:47:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:31.184071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7ebc9-8f32-4b21-ac0e-d1eb41a0b81a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:31.216803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bc46e-b3b0-439f-8412-1c53cec20511 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:31.250644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a740-24c8-4b22-a9e3-68e2239ce9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:31.251210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a740-24c8-4b22-a9e3-68e2239ce9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:31.251492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a740-24c8-4b22-a9e3-68e2239ce9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:31.251764 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447a740-24c8-4b22-a9e3-68e2239ce9b3 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:31.252390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447a740-24c8-4b22-a9e3-68e2239ce9b3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:31.252697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447a740-24c8-4b22-a9e3-68e2239ce9b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:31.254075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447a740-24c8-4b22-a9e3-68e2239ce9b3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:31.259234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 474/937] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Sep 16 07:47:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:31.260008 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a740-24c8-4b22-a9e3-68e2239ce9b3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:31.290984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:31.298222 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7ebc9-8f32-4b21-ac0e-d1eb41a0b81a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:31.333428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb3839-661a-4eb3-9116-f54bef9f76e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:31.334040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb3839-661a-4eb3-9116-f54bef9f76e6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:31.334317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb3839-661a-4eb3-9116-f54bef9f76e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:31.334585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb3839-661a-4eb3-9116-f54bef9f76e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:31.335302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb3839-661a-4eb3-9116-f54bef9f76e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:31.335606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb3839-661a-4eb3-9116-f54bef9f76e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:31.337175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb3839-661a-4eb3-9116-f54bef9f76e6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:31.354731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 464/938] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:31.354731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb3839-661a-4eb3-9116-f54bef9f76e6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:31.367909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447a740-24c8-4b22-a9e3-68e2239ce9b3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:31.378459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:31.403107 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac8d98-79d9-441f-849b-f902d5504c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:31.403748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac8d98-79d9-441f-849b-f902d5504c96 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:31.404232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac8d98-79d9-441f-849b-f902d5504c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:31.404673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac8d98-79d9-441f-849b-f902d5504c96 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:31.405548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac8d98-79d9-441f-849b-f902d5504c96 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:31.405855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac8d98-79d9-441f-849b-f902d5504c96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:31.407312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac8d98-79d9-441f-849b-f902d5504c96 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:31.417487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 475/939] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Sep 16 07:47:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:31.417487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac8d98-79d9-441f-849b-f902d5504c96 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:31.466580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb3839-661a-4eb3-9116-f54bef9f76e6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:31.503188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24612a2-ef0e-49ec-bd10-fa0af50e9cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:31.503718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24612a2-ef0e-49ec-bd10-fa0af50e9cfb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:31.504001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24612a2-ef0e-49ec-bd10-fa0af50e9cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:31.504269 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24612a2-ef0e-49ec-bd10-fa0af50e9cfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:31.504942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24612a2-ef0e-49ec-bd10-fa0af50e9cfb None admin] RBAC: Adding query filter params (interface=public, service_id=b3625e925f954627b47ab384ab006886, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:31.505238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24612a2-ef0e-49ec-bd10-fa0af50e9cfb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:31.506670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24612a2-ef0e-49ec-bd10-fa0af50e9cfb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:31.517022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 465/940] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Sep 16 07:47:31 2022] GET /identity/v3/endpoints?service_id=b3625e925f954627b47ab384ab006886&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:31.517022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24612a2-ef0e-49ec-bd10-fa0af50e9cfb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:31.521488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac8d98-79d9-441f-849b-f902d5504c96 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:31.534291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:31.558745 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea5cb8-6925-4a98-914c-d35d72d457bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:31.559399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea5cb8-6925-4a98-914c-d35d72d457bd None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:31.559677 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea5cb8-6925-4a98-914c-d35d72d457bd None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:31.559947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea5cb8-6925-4a98-914c-d35d72d457bd None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/92d4614a1449473a9b949d4192cd2a76/roles/cd21a152b61c4235ad578e9b7a40d857` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:31.562320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea5cb8-6925-4a98-914c-d35d72d457bd None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=92d4614a1449473a9b949d4192cd2a76, role_id=cd21a152b61c4235ad578e9b7a40d857)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:31.563836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea5cb8-6925-4a98-914c-d35d72d457bd None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:31.617058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24612a2-ef0e-49ec-bd10-fa0af50e9cfb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:31.652826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2f0bd-c0db-46dc-95ff-be8223ce9b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:31.653468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2f0bd-c0db-46dc-95ff-be8223ce9b36 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:31.653706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2f0bd-c0db-46dc-95ff-be8223ce9b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:31.653972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2f0bd-c0db-46dc-95ff-be8223ce9b36 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:31.654589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2f0bd-c0db-46dc-95ff-be8223ce9b36 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:31.654918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2f0bd-c0db-46dc-95ff-be8223ce9b36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:31.656295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2f0bd-c0db-46dc-95ff-be8223ce9b36 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:31.666977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 476/941] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:47:31 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/92d4614a1449473a9b949d4192cd2a76/roles/cd21a152b61c4235ad578e9b7a40d857 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:47:31.666977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea5cb8-6925-4a98-914c-d35d72d457bd None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:31.676798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 466/942] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Sep 16 07:47:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:31.677876 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2f0bd-c0db-46dc-95ff-be8223ce9b36 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:31.702988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:31.783993 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea5cb8-6925-4a98-914c-d35d72d457bd None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:31.800076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:31.822448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2f0bd-c0db-46dc-95ff-be8223ce9b36 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:31.834708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d378c-9637-4401-bb7f-14d26ce998ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:31.835308 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d378c-9637-4401-bb7f-14d26ce998ae None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:31.835584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d378c-9637-4401-bb7f-14d26ce998ae None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:31.836383 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d378c-9637-4401-bb7f-14d26ce998ae None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:31.837004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d378c-9637-4401-bb7f-14d26ce998ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:31.839414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d378c-9637-4401-bb7f-14d26ce998ae None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:31.847704 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 477/943] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Sep 16 07:47:31 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 07:47:31.879856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb648a-59e0-4131-9f47-c0338bdcfc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:31.880653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb648a-59e0-4131-9f47-c0338bdcfc51 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:31.880833 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb648a-59e0-4131-9f47-c0338bdcfc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:31.881103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb648a-59e0-4131-9f47-c0338bdcfc51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:31.881800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb648a-59e0-4131-9f47-c0338bdcfc51 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:31.883309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb648a-59e0-4131-9f47-c0338bdcfc51 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:32.012934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 467/944] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 07:47:31 2022] POST /identity/v3/endpoints => generated 347 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:47:32.408634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01d01d-6d71-4831-a2ec-c4a189157b2e None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:32.409152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01d01d-6d71-4831-a2ec-c4a189157b2e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:32.409229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01d01d-6d71-4831-a2ec-c4a189157b2e None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:32.410778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 478/945] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:32.416248 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b2062-7b1f-4e1c-89a9-a259017d61f1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:32.416536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b2062-7b1f-4e1c-89a9-a259017d61f1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:32.416814 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b2062-7b1f-4e1c-89a9-a259017d61f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:32.435550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c09b2062-7b1f-4e1c-89a9-a259017d61f1 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:32.437016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c09b2062-7b1f-4e1c-89a9-a259017d61f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:32.510789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 468/946] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:32.784429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01d01d-6d71-4831-a2ec-c4a189157b2e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:32.887740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:32.892148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01d01d-6d71-4831-a2ec-c4a189157b2e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:32.926097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0f2b7-aa89-4120-9235-d0b10e6ed7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:32.930344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0f2b7-aa89-4120-9235-d0b10e6ed7c2 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:32.930344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0f2b7-aa89-4120-9235-d0b10e6ed7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:32.930344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0f2b7-aa89-4120-9235-d0b10e6ed7c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:32.930344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0f2b7-aa89-4120-9235-d0b10e6ed7c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:32.930344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0f2b7-aa89-4120-9235-d0b10e6ed7c2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:32.932885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: WARNING keystone.server.flask.common [None req-ccf0f2b7-aa89-4120-9235-d0b10e6ed7c2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 16 07:47:32.975997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.sql.core [None req-ccf0f2b7-aa89-4120-9235-d0b10e6ed7c2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 16 07:47:32.975997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 16 07:47:32.975997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [parameters: {'id': '6bc115425e064324a4eafa3bfdbf5207', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 16 07:47:32.975997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81573) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-ccf0f2b7-aa89-4120-9235-d0b10e6ed7c2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6bc115425e064324a4eafa3bfdbf5207.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6bc115425e064324a4eafa3bfdbf5207. Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application conn.query(q) Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 07:47:32.994616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result.read() Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 16 07:47:32.995474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application next(self.gen) Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application yield resource Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application next(self.gen) Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.session.rollback() Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.force_reraise() Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise self.value Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application session.commit() Sep 16 07:47:32.996428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._prepare_impl() Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.session.flush() Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._flush(objects) Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application compat.raise_( Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application flush_context.execute() Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 16 07:47:32.998442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rec.execute(self) Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 16 07:47:33.000553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application conn.query(q) Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result.read() Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 07:47:33.002346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [parameters: {'id': '6bc115425e064324a4eafa3bfdbf5207', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:33.004396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:33.005669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:33.022987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:33.022987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 16 07:47:33.022987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 16 07:47:33.022987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:33.022987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:33.022987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 16 07:47:33.022987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 16 07:47:33.022987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6bc115425e064324a4eafa3bfdbf5207. Sep 16 07:47:33.022987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:33.022987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 479/947] 10.222.0.59 () {60 vars in 1140 bytes} [Fri Sep 16 07:47:32 2022] POST /identity/v3/projects => generated 262 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:33.023713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b2062-7b1f-4e1c-89a9-a259017d61f1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:33.071205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:33.109404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b2062-7b1f-4e1c-89a9-a259017d61f1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:33.144634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43942d-df8f-47ea-ba85-1b548a9e6c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:33.145267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43942d-df8f-47ea-ba85-1b548a9e6c93 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:33.145549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43942d-df8f-47ea-ba85-1b548a9e6c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:33.145818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43942d-df8f-47ea-ba85-1b548a9e6c93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:33.153193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43942d-df8f-47ea-ba85-1b548a9e6c93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:33.154651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43942d-df8f-47ea-ba85-1b548a9e6c93 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ee43942d-df8f-47ea-ba85-1b548a9e6c93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:33.166554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:33.168363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:33.169730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:33.169730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:33.169730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:33.169730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:33.169730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:33.169730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:33.169730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:33.169730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:33.169730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 469/948] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:33.173926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0f2b7-aa89-4120-9235-d0b10e6ed7c2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:33.278211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0f2b7-aa89-4120-9235-d0b10e6ed7c2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:33.315247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ddbcc-e585-4882-8a48-2bbef2da9ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:33.315899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ddbcc-e585-4882-8a48-2bbef2da9ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:33.316186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ddbcc-e585-4882-8a48-2bbef2da9ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:33.316461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ddbcc-e585-4882-8a48-2bbef2da9ed4 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:33.317124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ddbcc-e585-4882-8a48-2bbef2da9ed4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:33.317428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ddbcc-e585-4882-8a48-2bbef2da9ed4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:33.319089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ddbcc-e585-4882-8a48-2bbef2da9ed4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:33.340295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 480/949] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:33 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:33.969440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc0db0-fd42-4e30-b7c2-5ae1347b69bc None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:33.970162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc0db0-fd42-4e30-b7c2-5ae1347b69bc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:33.970162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc0db0-fd42-4e30-b7c2-5ae1347b69bc None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:33.971725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 470/950] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:33.977345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fa4e8-f99b-48c2-8612-03e71a373a4e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:33.977852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fa4e8-f99b-48c2-8612-03e71a373a4e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:33.978019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fa4e8-f99b-48c2-8612-03e71a373a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:33.997008 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9f0fa4e8-f99b-48c2-8612-03e71a373a4e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:33.998436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9f0fa4e8-f99b-48c2-8612-03e71a373a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:34.008578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:34.062125 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c791010-c518-4056-9ec5-f18680738a74 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:34.062515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c791010-c518-4056-9ec5-f18680738a74 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:34.062901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c791010-c518-4056-9ec5-f18680738a74 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:34.064518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 471/951] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:34.073229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8131d4-b95e-4492-b96b-b6a4ac580af2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:34.073229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8131d4-b95e-4492-b96b-b6a4ac580af2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:34.073229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8131d4-b95e-4492-b96b-b6a4ac580af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:34.073528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 481/952] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:34.089283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7d8131d4-b95e-4492-b96b-b6a4ac580af2 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:34.090469 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7d8131d4-b95e-4492-b96b-b6a4ac580af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:34.160042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 472/953] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:34.283302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efd99b-94f4-43c1-bc11-0de8997e66ce None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:34.283487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efd99b-94f4-43c1-bc11-0de8997e66ce None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:34.283774 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efd99b-94f4-43c1-bc11-0de8997e66ce None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:34.285418 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 482/954] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:34.291622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ae3bc-869a-4086-95ab-0301a71e0c11 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:34.291926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ae3bc-869a-4086-95ab-0301a71e0c11 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:34.292177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ae3bc-869a-4086-95ab-0301a71e0c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:34.312173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fc1ae3bc-869a-4086-95ab-0301a71e0c11 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:34.313374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fc1ae3bc-869a-4086-95ab-0301a71e0c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:34.361873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:34.369479 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efd99b-94f4-43c1-bc11-0de8997e66ce None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:34.387718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 473/955] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:34.443427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ae3bc-869a-4086-95ab-0301a71e0c11 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:34.480350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efd99b-94f4-43c1-bc11-0de8997e66ce None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:34.524055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d5b71-608a-40de-a04d-2f40378de2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:34.525592 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d5b71-608a-40de-a04d-2f40378de2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:34.526339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d5b71-608a-40de-a04d-2f40378de2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:34.526851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d5b71-608a-40de-a04d-2f40378de2f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:34.541158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d5b71-608a-40de-a04d-2f40378de2f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b66d5b71-608a-40de-a04d-2f40378de2f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:34.548056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:34.549050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:34.550057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:34.555067 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 483/956] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Sep 16 07:47:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:34.559064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d5b71-608a-40de-a04d-2f40378de2f5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:34.564400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:34.570386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ae3bc-869a-4086-95ab-0301a71e0c11 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:34.614654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2072f-7225-4a1e-84b0-49eafe5f5db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:34.615925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2072f-7225-4a1e-84b0-49eafe5f5db2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:34.616731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2072f-7225-4a1e-84b0-49eafe5f5db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:34.617748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2072f-7225-4a1e-84b0-49eafe5f5db2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:34.618704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2072f-7225-4a1e-84b0-49eafe5f5db2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:34.620861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2072f-7225-4a1e-84b0-49eafe5f5db2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-44d2072f-7225-4a1e-84b0-49eafe5f5db2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:34.632932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:34.634024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:34.634934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:34.640954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 474/957] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Sep 16 07:47:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:34.642565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2072f-7225-4a1e-84b0-49eafe5f5db2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:34.685258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d5b71-608a-40de-a04d-2f40378de2f5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:34.697818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:34.737979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881defa-7677-447c-9375-18eafdc7c7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:34.737979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881defa-7677-447c-9375-18eafdc7c7ca None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:34.737979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881defa-7677-447c-9375-18eafdc7c7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:34.737979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881defa-7677-447c-9375-18eafdc7c7ca None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:34.738395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881defa-7677-447c-9375-18eafdc7c7ca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:34.738395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881defa-7677-447c-9375-18eafdc7c7ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:34.739961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881defa-7677-447c-9375-18eafdc7c7ca None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:34.749731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2072f-7225-4a1e-84b0-49eafe5f5db2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:34.755002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 484/958] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Sep 16 07:47:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:34.756262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881defa-7677-447c-9375-18eafdc7c7ca None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:34.787796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699b366-3671-4484-9194-0c3643ba596d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:34.788413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699b366-3671-4484-9194-0c3643ba596d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:34.788695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699b366-3671-4484-9194-0c3643ba596d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:34.789015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699b366-3671-4484-9194-0c3643ba596d None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:34.790138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699b366-3671-4484-9194-0c3643ba596d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:34.790442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699b366-3671-4484-9194-0c3643ba596d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:34.791913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699b366-3671-4484-9194-0c3643ba596d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:34.796532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 475/959] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Sep 16 07:47:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:34.797312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699b366-3671-4484-9194-0c3643ba596d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:34.827668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:34.865295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881defa-7677-447c-9375-18eafdc7c7ca None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:34.901559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c07dd-1e67-4713-b237-c1ac55aeb174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:34.902187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c07dd-1e67-4713-b237-c1ac55aeb174 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:34.902463 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c07dd-1e67-4713-b237-c1ac55aeb174 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:34.902762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c07dd-1e67-4713-b237-c1ac55aeb174 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:34.903339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c07dd-1e67-4713-b237-c1ac55aeb174 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:34.904682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c07dd-1e67-4713-b237-c1ac55aeb174 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:34.905890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699b366-3671-4484-9194-0c3643ba596d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-954c07dd-1e67-4713-b237-c1ac55aeb174 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:34.913536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:34.915900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:34.916987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 485/960] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:34.917911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c07dd-1e67-4713-b237-c1ac55aeb174 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:34.924132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:34.945198 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d429c4-61cc-4195-9a1d-933759b68a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:34.945826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d429c4-61cc-4195-9a1d-933759b68a72 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:34.946129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d429c4-61cc-4195-9a1d-933759b68a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:34.946399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d429c4-61cc-4195-9a1d-933759b68a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:34.959590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d429c4-61cc-4195-9a1d-933759b68a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:34.961560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d429c4-61cc-4195-9a1d-933759b68a72 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-18d429c4-61cc-4195-9a1d-933759b68a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:34.975551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:34.982947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:34.983865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:34.983865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:34.983865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:34.983865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 476/961] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:34.983865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d429c4-61cc-4195-9a1d-933759b68a72 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:35.033512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c07dd-1e67-4713-b237-c1ac55aeb174 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:35.063766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:35.069941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cac7a-5b5a-449f-803a-0ab031be6543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:35.073853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cac7a-5b5a-449f-803a-0ab031be6543 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:35.073853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cac7a-5b5a-449f-803a-0ab031be6543 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:35.073853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cac7a-5b5a-449f-803a-0ab031be6543 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:35.073853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cac7a-5b5a-449f-803a-0ab031be6543 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:35.073853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cac7a-5b5a-449f-803a-0ab031be6543 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:35.073853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cac7a-5b5a-449f-803a-0ab031be6543 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:35.078754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 486/962] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Sep 16 07:47:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:35.084195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cac7a-5b5a-449f-803a-0ab031be6543 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:35.084774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d429c4-61cc-4195-9a1d-933759b68a72 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:35.119450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc23949-8771-4473-a512-f43eca2dcc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:35.120063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc23949-8771-4473-a512-f43eca2dcc25 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:35.120340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc23949-8771-4473-a512-f43eca2dcc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:35.120604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc23949-8771-4473-a512-f43eca2dcc25 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:35.121251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc23949-8771-4473-a512-f43eca2dcc25 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:35.121551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc23949-8771-4473-a512-f43eca2dcc25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:35.122936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc23949-8771-4473-a512-f43eca2dcc25 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:35.127703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 477/963] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:35.128257 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc23949-8771-4473-a512-f43eca2dcc25 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:35.157477 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:35.199838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cac7a-5b5a-449f-803a-0ab031be6543 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:35.233657 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc23949-8771-4473-a512-f43eca2dcc25 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:35.236516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6960114-2eb7-4537-8fe4-ea0e08de8aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:35.236674 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6960114-2eb7-4537-8fe4-ea0e08de8aaf None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:35.236895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6960114-2eb7-4537-8fe4-ea0e08de8aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:35.237168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6960114-2eb7-4537-8fe4-ea0e08de8aaf None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:35.237803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6960114-2eb7-4537-8fe4-ea0e08de8aaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:35.238102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6960114-2eb7-4537-8fe4-ea0e08de8aaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:35.239482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6960114-2eb7-4537-8fe4-ea0e08de8aaf None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:35.256375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 487/964] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:35.257250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6960114-2eb7-4537-8fe4-ea0e08de8aaf None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:35.269652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ef972-f099-4781-b208-f80da949f434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:35.270266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ef972-f099-4781-b208-f80da949f434 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:35.270557 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ef972-f099-4781-b208-f80da949f434 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:35.270875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ef972-f099-4781-b208-f80da949f434 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:35.271550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ef972-f099-4781-b208-f80da949f434 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:35.272957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ef972-f099-4781-b208-f80da949f434 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:35.312635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:35.331767 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 478/965] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 07:47:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:47:35.331767 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ef972-f099-4781-b208-f80da949f434 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:35.360297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6960114-2eb7-4537-8fe4-ea0e08de8aaf None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:35.411920 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecc6b2-a021-4263-a14f-7123d5aa9b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:35.413033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecc6b2-a021-4263-a14f-7123d5aa9b6f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:35.413782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecc6b2-a021-4263-a14f-7123d5aa9b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:35.414278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecc6b2-a021-4263-a14f-7123d5aa9b6f None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:35.415151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecc6b2-a021-4263-a14f-7123d5aa9b6f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:35.415746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecc6b2-a021-4263-a14f-7123d5aa9b6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:35.417723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecc6b2-a021-4263-a14f-7123d5aa9b6f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:35.431341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 488/966] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:35.447063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:35.466835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ef972-f099-4781-b208-f80da949f434 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:35.509519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2de17-ebd0-4efe-9eda-129348f91a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:35.510191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2de17-ebd0-4efe-9eda-129348f91a58 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:35.510514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2de17-ebd0-4efe-9eda-129348f91a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:35.510818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2de17-ebd0-4efe-9eda-129348f91a58 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:35.520049 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2de17-ebd0-4efe-9eda-129348f91a58 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:35.522115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2de17-ebd0-4efe-9eda-129348f91a58 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3ad2de17-ebd0-4efe-9eda-129348f91a58 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:35.543920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:35.545015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:35.545983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:35.545983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:35.545983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:35.545983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:35.545983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:35.545983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:35.545983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:35.545983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:35.545983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 07:47:35.545983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:35.548558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 479/967] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Sep 16 07:47:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:35.557929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecc6b2-a021-4263-a14f-7123d5aa9b6f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:35.597771 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83cfdf-e281-440a-9db0-c3a2b3bbb5b1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:35.597989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83cfdf-e281-440a-9db0-c3a2b3bbb5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:35.598265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83cfdf-e281-440a-9db0-c3a2b3bbb5b1 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:35.600732 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 480/968] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:35.606498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90164c64-a8f9-4516-b7e1-34724f40a4b6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:35.606818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90164c64-a8f9-4516-b7e1-34724f40a4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:35.607084 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90164c64-a8f9-4516-b7e1-34724f40a4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:35.628104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-90164c64-a8f9-4516-b7e1-34724f40a4b6 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:35.629298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-90164c64-a8f9-4516-b7e1-34724f40a4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:35.679897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecc6b2-a021-4263-a14f-7123d5aa9b6f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:35.698145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:35.708698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 481/969] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:35.725834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f625e-6dfc-4229-9d38-b63a29251089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:35.726580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f625e-6dfc-4229-9d38-b63a29251089 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:35.726893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f625e-6dfc-4229-9d38-b63a29251089 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:35.727219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f625e-6dfc-4229-9d38-b63a29251089 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:35.728012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f625e-6dfc-4229-9d38-b63a29251089 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:35.728315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f625e-6dfc-4229-9d38-b63a29251089 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:35.729790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f625e-6dfc-4229-9d38-b63a29251089 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:35.753741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 489/970] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Sep 16 07:47:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:35.755120 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90164c64-a8f9-4516-b7e1-34724f40a4b6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:35.835123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:35.860896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90164c64-a8f9-4516-b7e1-34724f40a4b6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:35.897183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba1bf1-8176-4fc1-bc22-03f032fd9db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:35.897793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba1bf1-8176-4fc1-bc22-03f032fd9db1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:35.898091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba1bf1-8176-4fc1-bc22-03f032fd9db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:35.898362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba1bf1-8176-4fc1-bc22-03f032fd9db1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:35.971055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba1bf1-8176-4fc1-bc22-03f032fd9db1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:35.972509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba1bf1-8176-4fc1-bc22-03f032fd9db1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:35.993953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f625e-6dfc-4229-9d38-b63a29251089 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:36.062643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:36.091560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 482/971] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:35 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:47:36.099560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f625e-6dfc-4229-9d38-b63a29251089 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:36.100517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba1bf1-8176-4fc1-bc22-03f032fd9db1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:36.135614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddeaf7f-aa7a-4a4c-858e-d4a7b9e4f852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:36.136207 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddeaf7f-aa7a-4a4c-858e-d4a7b9e4f852 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:36.136489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddeaf7f-aa7a-4a4c-858e-d4a7b9e4f852 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:36.136764 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddeaf7f-aa7a-4a4c-858e-d4a7b9e4f852 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:36.137408 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddeaf7f-aa7a-4a4c-858e-d4a7b9e4f852 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:36.138860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddeaf7f-aa7a-4a4c-858e-d4a7b9e4f852 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.sql.core [None req-5ddeaf7f-aa7a-4a4c-858e-d4a7b9e4f852 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [parameters: {'user_id': '98bc750aee3c4d1e928d61946511f724', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81573) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-5ddeaf7f-aa7a-4a4c-858e-d4a7b9e4f852 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 07:47:36.190705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application conn.query(q) Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result.read() Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:36.191568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application next(self.gen) Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application yield resource Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application next(self.gen) Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.session.rollback() Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.force_reraise() Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 16 07:47:36.192502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise self.value Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application session.commit() Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._prepare_impl() Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.session.flush() Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._flush(objects) Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 16 07:47:36.193466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application compat.raise_( Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application flush_context.execute() Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rec.execute(self) Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 16 07:47:36.194319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 07:47:36.195847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application conn.query(q) Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result.read() Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 16 07:47:36.196729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [parameters: {'user_id': '98bc750aee3c4d1e928d61946511f724', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:36.197480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:36.198411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 490/972] 10.222.0.59 () {60 vars in 1131 bytes} [Fri Sep 16 07:47:35 2022] POST /identity/v3/users => generated 160 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:36.207746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddeaf7f-aa7a-4a4c-858e-d4a7b9e4f852 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:36.216469 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba1bf1-8176-4fc1-bc22-03f032fd9db1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:36.245971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:36.260963 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e05bd4-7246-4f55-82b0-b3639415fa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:36.261603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e05bd4-7246-4f55-82b0-b3639415fa3a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:36.261879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e05bd4-7246-4f55-82b0-b3639415fa3a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:36.262155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e05bd4-7246-4f55-82b0-b3639415fa3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:36.274554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e05bd4-7246-4f55-82b0-b3639415fa3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:36.275996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e05bd4-7246-4f55-82b0-b3639415fa3a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-66e05bd4-7246-4f55-82b0-b3639415fa3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:36.291155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:36.297564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:36.298474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:36.298474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:36.298474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:36.298474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 483/973] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:36.298474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e05bd4-7246-4f55-82b0-b3639415fa3a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:36.324822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddeaf7f-aa7a-4a4c-858e-d4a7b9e4f852 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:36.328040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:36.363310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bb4e8-93f8-4736-b965-3be7be997806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:36.363945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bb4e8-93f8-4736-b965-3be7be997806 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:36.364246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bb4e8-93f8-4736-b965-3be7be997806 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:36.364780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bb4e8-93f8-4736-b965-3be7be997806 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:36.373564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bb4e8-93f8-4736-b965-3be7be997806 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:36.383193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bb4e8-93f8-4736-b965-3be7be997806 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-507bb4e8-93f8-4736-b965-3be7be997806 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:36.388263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:36.389229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:36.390215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:36.390215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:36.390215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:36.390215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:36.390215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:36.390215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:36.390215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:36.390215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:36.390215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 07:47:36.390215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:36.394405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 491/974] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:47:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:36.395413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bb4e8-93f8-4736-b965-3be7be997806 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:36.409485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e05bd4-7246-4f55-82b0-b3639415fa3a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:36.445824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e4d5b-f294-4e1f-bfae-5e8258f3d19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:36.446429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e4d5b-f294-4e1f-bfae-5e8258f3d19e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:36.446734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e4d5b-f294-4e1f-bfae-5e8258f3d19e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:36.447006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e4d5b-f294-4e1f-bfae-5e8258f3d19e None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:36.447645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e4d5b-f294-4e1f-bfae-5e8258f3d19e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:36.447942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e4d5b-f294-4e1f-bfae-5e8258f3d19e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:36.449311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e4d5b-f294-4e1f-bfae-5e8258f3d19e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:36.466709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 484/975] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:36.467936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e4d5b-f294-4e1f-bfae-5e8258f3d19e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:36.496430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:36.503979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bb4e8-93f8-4736-b965-3be7be997806 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:36.540360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa217af9-a42f-452c-8353-b62bd181b0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:36.541267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa217af9-a42f-452c-8353-b62bd181b0cb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:36.541702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa217af9-a42f-452c-8353-b62bd181b0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:36.542130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa217af9-a42f-452c-8353-b62bd181b0cb None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:36.543032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa217af9-a42f-452c-8353-b62bd181b0cb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:36.543488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa217af9-a42f-452c-8353-b62bd181b0cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:36.545056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa217af9-a42f-452c-8353-b62bd181b0cb None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:36.555715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:36.571678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e4d5b-f294-4e1f-bfae-5e8258f3d19e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:36.591085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 492/976] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Sep 16 07:47:36 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:36.609026 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1bddf-5cca-46f3-b26a-a67a58088b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:36.609680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1bddf-5cca-46f3-b26a-a67a58088b1b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:36.609955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1bddf-5cca-46f3-b26a-a67a58088b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:36.610221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1bddf-5cca-46f3-b26a-a67a58088b1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:36.619019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1bddf-5cca-46f3-b26a-a67a58088b1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:36.620547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1bddf-5cca-46f3-b26a-a67a58088b1b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a3e1bddf-5cca-46f3-b26a-a67a58088b1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:36.633274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:36.634458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:36.635589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:36.635589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:36.635589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:36.635589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:36.635589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:36.635589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:36.635589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:36.635589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:36.638880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 485/977] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:36.640729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa217af9-a42f-452c-8353-b62bd181b0cb None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:36.763633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa217af9-a42f-452c-8353-b62bd181b0cb None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:36.806314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee75b5-52ec-4d71-8fa4-852cb2c80b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:36.807353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee75b5-52ec-4d71-8fa4-852cb2c80b4b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:36.807867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee75b5-52ec-4d71-8fa4-852cb2c80b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:36.808346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee75b5-52ec-4d71-8fa4-852cb2c80b4b None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:36.809264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee75b5-52ec-4d71-8fa4-852cb2c80b4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:36.809775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee75b5-52ec-4d71-8fa4-852cb2c80b4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:36.811701 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee75b5-52ec-4d71-8fa4-852cb2c80b4b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:36.825028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 493/978] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:36.830932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1bddf-5cca-46f3-b26a-a67a58088b1b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:36.939483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1bddf-5cca-46f3-b26a-a67a58088b1b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:36.975166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7254c22-2afa-4fcd-a7fd-13f127bab08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:36.975774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7254c22-2afa-4fcd-a7fd-13f127bab08b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:36.976106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7254c22-2afa-4fcd-a7fd-13f127bab08b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:36.976442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7254c22-2afa-4fcd-a7fd-13f127bab08b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:36.977172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7254c22-2afa-4fcd-a7fd-13f127bab08b None admin] RBAC: Adding query filter params (role.id=cd21a152b61c4235ad578e9b7a40d857, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=92d4614a1449473a9b949d4192cd2a76) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:36.977482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7254c22-2afa-4fcd-a7fd-13f127bab08b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:36.978937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7254c22-2afa-4fcd-a7fd-13f127bab08b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:36.985068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 486/979] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:36 2022] GET /identity/v3/role_assignments?user.id=92d4614a1449473a9b949d4192cd2a76&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=cd21a152b61c4235ad578e9b7a40d857 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:37.637922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b08806-b4df-4d53-85fa-3e5f88d8ec5a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:37.638471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b08806-b4df-4d53-85fa-3e5f88d8ec5a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:37.638471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b08806-b4df-4d53-85fa-3e5f88d8ec5a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:37.639975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 494/980] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:37.645042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716f640-0fdc-4696-87cc-0181b7645152 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:37.645424 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716f640-0fdc-4696-87cc-0181b7645152 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:37.645599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716f640-0fdc-4696-87cc-0181b7645152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:37.664582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0716f640-0fdc-4696-87cc-0181b7645152 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:37.665776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0716f640-0fdc-4696-87cc-0181b7645152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:37.668745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:37.682398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af273d-0e84-4989-9ef4-062996e879f5 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:37.682708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af273d-0e84-4989-9ef4-062996e879f5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:37.695034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af273d-0e84-4989-9ef4-062996e879f5 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:37.695034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 495/981] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:37.695034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce4de5-fbc5-4566-882b-dbabda52c3c8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:37.695034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce4de5-fbc5-4566-882b-dbabda52c3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:37.695034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce4de5-fbc5-4566-882b-dbabda52c3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:37.709891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-29ce4de5-fbc5-4566-882b-dbabda52c3c8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:37.712284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-29ce4de5-fbc5-4566-882b-dbabda52c3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:37.721992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:37.742727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 487/982] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:37.785810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 496/983] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:38.006486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716f640-0fdc-4696-87cc-0181b7645152 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:38.052630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce4de5-fbc5-4566-882b-dbabda52c3c8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:38.108067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716f640-0fdc-4696-87cc-0181b7645152 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:38.143280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b584cb4-5447-4849-9e25-26f65cd402f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:38.143888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b584cb4-5447-4849-9e25-26f65cd402f5 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:38.144167 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b584cb4-5447-4849-9e25-26f65cd402f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:38.144439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b584cb4-5447-4849-9e25-26f65cd402f5 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:38.145109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b584cb4-5447-4849-9e25-26f65cd402f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:38.146476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b584cb4-5447-4849-9e25-26f65cd402f5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:38.161606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce4de5-fbc5-4566-882b-dbabda52c3c8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:38.196508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e4224-de0a-466c-a165-683fd6f67475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:38.197135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e4224-de0a-466c-a165-683fd6f67475 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:38.197456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e4224-de0a-466c-a165-683fd6f67475 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:38.197727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e4224-de0a-466c-a165-683fd6f67475 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:38.198463 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e4224-de0a-466c-a165-683fd6f67475 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:38.199862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e4224-de0a-466c-a165-683fd6f67475 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d58e4224-de0a-466c-a165-683fd6f67475 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:38.208549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:38.209543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:38.210388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 497/984] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Sep 16 07:47:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:38.213703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:38.216387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e4224-de0a-466c-a165-683fd6f67475 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:38.219230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 488/985] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:47:38 2022] POST /identity/v3/services => generated 242 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:47:38.230203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:38.333809 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e4224-de0a-466c-a165-683fd6f67475 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:38.376771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efcb58-5995-4816-9335-95e3e8d7a75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:38.377394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efcb58-5995-4816-9335-95e3e8d7a75a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:38.377675 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efcb58-5995-4816-9335-95e3e8d7a75a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:38.377951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efcb58-5995-4816-9335-95e3e8d7a75a None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:38.378585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efcb58-5995-4816-9335-95e3e8d7a75a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:38.378913 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efcb58-5995-4816-9335-95e3e8d7a75a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:38.380284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efcb58-5995-4816-9335-95e3e8d7a75a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:38.390748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 498/986] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Sep 16 07:47:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:38.392310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b584cb4-5447-4849-9e25-26f65cd402f5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:38.497219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b584cb4-5447-4849-9e25-26f65cd402f5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:38.532074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c714c5-26b9-4b1d-a0bd-46fc4785b8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:38.532689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c714c5-26b9-4b1d-a0bd-46fc4785b8df None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:38.532970 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c714c5-26b9-4b1d-a0bd-46fc4785b8df None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:38.533244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c714c5-26b9-4b1d-a0bd-46fc4785b8df None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:38.533881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c714c5-26b9-4b1d-a0bd-46fc4785b8df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:38.534199 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c714c5-26b9-4b1d-a0bd-46fc4785b8df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:38.535593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c714c5-26b9-4b1d-a0bd-46fc4785b8df None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:38.541402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 489/987] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Sep 16 07:47:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:38.547264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efcb58-5995-4816-9335-95e3e8d7a75a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:38.614145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:38.648814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efcb58-5995-4816-9335-95e3e8d7a75a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:38.682875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409ebe4-99f9-499d-acfb-e6c5df0ecb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:38.683485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409ebe4-99f9-499d-acfb-e6c5df0ecb7a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:38.683765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409ebe4-99f9-499d-acfb-e6c5df0ecb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:38.684040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409ebe4-99f9-499d-acfb-e6c5df0ecb7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:38.684749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409ebe4-99f9-499d-acfb-e6c5df0ecb7a None admin] RBAC: Adding query filter params (interface=admin, service_id=b3625e925f954627b47ab384ab006886, region_id=RegionOne) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:38.685698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409ebe4-99f9-499d-acfb-e6c5df0ecb7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:38.686673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409ebe4-99f9-499d-acfb-e6c5df0ecb7a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:38.691989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 499/988] 10.222.0.59 () {56 vars in 1249 bytes} [Fri Sep 16 07:47:38 2022] GET /identity/v3/endpoints?service_id=b3625e925f954627b47ab384ab006886&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:38.697656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c714c5-26b9-4b1d-a0bd-46fc4785b8df None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:38.772779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:38.812082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c714c5-26b9-4b1d-a0bd-46fc4785b8df None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:38.845700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe810ab9-9e6f-4bf9-98ab-655e30165c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:38.846304 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe810ab9-9e6f-4bf9-98ab-655e30165c81 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:38.846585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe810ab9-9e6f-4bf9-98ab-655e30165c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:38.846897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe810ab9-9e6f-4bf9-98ab-655e30165c81 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:38.847501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe810ab9-9e6f-4bf9-98ab-655e30165c81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:38.848861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe810ab9-9e6f-4bf9-98ab-655e30165c81 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:38.855149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 490/989] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Sep 16 07:47:38 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 07:47:39.155413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab4753-6291-417d-a40f-690e95e9711a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:39.155820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab4753-6291-417d-a40f-690e95e9711a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:39.156012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab4753-6291-417d-a40f-690e95e9711a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:39.157475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 500/990] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:39.162354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9431c1-0701-4890-8457-b7dbe25e30be None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:39.162639 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9431c1-0701-4890-8457-b7dbe25e30be None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:39.162949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9431c1-0701-4890-8457-b7dbe25e30be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:39.181455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9b9431c1-0701-4890-8457-b7dbe25e30be None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:39.182669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9b9431c1-0701-4890-8457-b7dbe25e30be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:39.250092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 491/991] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:39.514218 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab4753-6291-417d-a40f-690e95e9711a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:39.619737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab4753-6291-417d-a40f-690e95e9711a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:39.647240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:39.653287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2bc2b-4b85-469f-ada4-f984d449d4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:39.653953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2bc2b-4b85-469f-ada4-f984d449d4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:39.654200 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2bc2b-4b85-469f-ada4-f984d449d4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:39.654474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2bc2b-4b85-469f-ada4-f984d449d4d0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:39.655083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2bc2b-4b85-469f-ada4-f984d449d4d0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:39.656478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2bc2b-4b85-469f-ada4-f984d449d4d0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-22c2bc2b-4b85-469f-ada4-f984d449d4d0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:39.665655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:39.670750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:39.671817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 501/992] 10.222.0.59 () {56 vars in 1105 bytes} [Fri Sep 16 07:47:39 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:39.673731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9431c1-0701-4890-8457-b7dbe25e30be None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:39.743220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:39.782706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9431c1-0701-4890-8457-b7dbe25e30be None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:39.816869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7343bc1-7800-4368-9fa7-18ad48006563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:39.817474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7343bc1-7800-4368-9fa7-18ad48006563 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:39.817751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7343bc1-7800-4368-9fa7-18ad48006563 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:39.818019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7343bc1-7800-4368-9fa7-18ad48006563 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:39.818643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7343bc1-7800-4368-9fa7-18ad48006563 None admin] RBAC: Adding query filter params (name=image) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:39.818968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7343bc1-7800-4368-9fa7-18ad48006563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:39.820351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7343bc1-7800-4368-9fa7-18ad48006563 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:39.825240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 492/993] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:39.830698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2bc2b-4b85-469f-ada4-f984d449d4d0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:39.934254 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2bc2b-4b85-469f-ada4-f984d449d4d0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:39.969469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b1d68-3f66-4647-935a-a6011c73c007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:39.970081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b1d68-3f66-4647-935a-a6011c73c007 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:39.970364 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b1d68-3f66-4647-935a-a6011c73c007 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:39.970639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b1d68-3f66-4647-935a-a6011c73c007 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:39.971300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b1d68-3f66-4647-935a-a6011c73c007 None admin] RBAC: Adding query filter params (type=image) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:39.971609 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b1d68-3f66-4647-935a-a6011c73c007 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:39.972976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b1d68-3f66-4647-935a-a6011c73c007 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:39.988201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 502/994] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:39 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:40.409715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd4bef-c9ee-4223-9c4f-cad10db82713 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:40.410090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd4bef-c9ee-4223-9c4f-cad10db82713 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:40.410312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd4bef-c9ee-4223-9c4f-cad10db82713 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:40.411779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 493/995] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:40.416836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db371631-6106-4379-a174-796d1f63f2a9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:40.417111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db371631-6106-4379-a174-796d1f63f2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:40.417385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db371631-6106-4379-a174-796d1f63f2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:40.435349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-db371631-6106-4379-a174-796d1f63f2a9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:40.436806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-db371631-6106-4379-a174-796d1f63f2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:40.492855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:40.506413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 503/996] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:40 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:40.775541 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd4bef-c9ee-4223-9c4f-cad10db82713 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:40.880931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd4bef-c9ee-4223-9c4f-cad10db82713 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:40.906653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:40.919718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25957a4a-4cbe-410d-8857-3a59ab12ecf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:40.920379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25957a4a-4cbe-410d-8857-3a59ab12ecf7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:40.920654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25957a4a-4cbe-410d-8857-3a59ab12ecf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:40.920989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25957a4a-4cbe-410d-8857-3a59ab12ecf7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:40.921589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25957a4a-4cbe-410d-8857-3a59ab12ecf7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:40.923147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25957a4a-4cbe-410d-8857-3a59ab12ecf7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-25957a4a-4cbe-410d-8857-3a59ab12ecf7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:40.932966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:40.934118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 07:47:40.935594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:40.939158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 494/997] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:40.940570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db371631-6106-4379-a174-796d1f63f2a9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:41.016105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f3317-0dab-4d4b-9070-1436b0c1848d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:41.016388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f3317-0dab-4d4b-9070-1436b0c1848d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:41.016593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f3317-0dab-4d4b-9070-1436b0c1848d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:41.018047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 495/998] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:41.022775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301f29a-8bd9-4181-bd3f-23aa56a08c67 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:41.023092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301f29a-8bd9-4181-bd3f-23aa56a08c67 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:41.023366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301f29a-8bd9-4181-bd3f-23aa56a08c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:41.042126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b301f29a-8bd9-4181-bd3f-23aa56a08c67 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:41.043361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b301f29a-8bd9-4181-bd3f-23aa56a08c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:41.044301 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db371631-6106-4379-a174-796d1f63f2a9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:41.081306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d711c8-6be3-49cb-b13f-abb773429e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:41.081971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d711c8-6be3-49cb-b13f-abb773429e1c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:41.082251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d711c8-6be3-49cb-b13f-abb773429e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:41.082523 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d711c8-6be3-49cb-b13f-abb773429e1c None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:41.083281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d711c8-6be3-49cb-b13f-abb773429e1c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:41.083591 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d711c8-6be3-49cb-b13f-abb773429e1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:41.084949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d711c8-6be3-49cb-b13f-abb773429e1c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:41.094712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 504/999] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:41.095561 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d711c8-6be3-49cb-b13f-abb773429e1c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:41.112006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 496/1000] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:41.149242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:41.199620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d711c8-6be3-49cb-b13f-abb773429e1c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:41.232986 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f4744-b00b-42a7-99bc-5c46989abd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:41.233585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f4744-b00b-42a7-99bc-5c46989abd2e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:41.233861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f4744-b00b-42a7-99bc-5c46989abd2e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:41.234130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f4744-b00b-42a7-99bc-5c46989abd2e None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:41.234773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f4744-b00b-42a7-99bc-5c46989abd2e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:41.235076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f4744-b00b-42a7-99bc-5c46989abd2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:41.236420 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f4744-b00b-42a7-99bc-5c46989abd2e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:41.241347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 505/1001] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:41.247451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301f29a-8bd9-4181-bd3f-23aa56a08c67 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:41.346606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301f29a-8bd9-4181-bd3f-23aa56a08c67 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:41.363603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:41.379453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f4744-b00b-42a7-99bc-5c46989abd2e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:41.381363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d300a-66d7-4a6a-be6b-b8ebce45858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:41.381975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d300a-66d7-4a6a-be6b-b8ebce45858b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:41.382573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d300a-66d7-4a6a-be6b-b8ebce45858b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:41.382881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d300a-66d7-4a6a-be6b-b8ebce45858b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:41.384520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d300a-66d7-4a6a-be6b-b8ebce45858b None admin] RBAC: Adding query filter params (interface=public, service_id=f850581fd442427fb0042a3e0c159c02, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:41.384827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d300a-66d7-4a6a-be6b-b8ebce45858b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:41.386217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d300a-66d7-4a6a-be6b-b8ebce45858b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:41.393547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 497/1002] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Sep 16 07:47:41 2022] GET /identity/v3/endpoints?service_id=f850581fd442427fb0042a3e0c159c02&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:41.399079 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d300a-66d7-4a6a-be6b-b8ebce45858b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:41.487177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f4744-b00b-42a7-99bc-5c46989abd2e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:41.499201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d300a-66d7-4a6a-be6b-b8ebce45858b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:41.504732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:41.525465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7de82c-0e86-47c0-81a3-08c19e244e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:41.526103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7de82c-0e86-47c0-81a3-08c19e244e50 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:41.526382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7de82c-0e86-47c0-81a3-08c19e244e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:41.526659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7de82c-0e86-47c0-81a3-08c19e244e50 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:41.527268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7de82c-0e86-47c0-81a3-08c19e244e50 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:41.528620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7de82c-0e86-47c0-81a3-08c19e244e50 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:41.535508 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f8cdd-cb38-4310-a846-7d90445e913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:41.536125 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f8cdd-cb38-4310-a846-7d90445e913b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:41.536397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f8cdd-cb38-4310-a846-7d90445e913b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:41.536673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f8cdd-cb38-4310-a846-7d90445e913b None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:41.537281 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f8cdd-cb38-4310-a846-7d90445e913b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-fb7de82c-0e86-47c0-81a3-08c19e244e50 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:41.538367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:41.540199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 07:47:41.541055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:41.541852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f8cdd-cb38-4310-a846-7d90445e913b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:41.555128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 498/1003] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Sep 16 07:47:41 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 07:47:41.555359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 506/1004] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Sep 16 07:47:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:41.555456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f8cdd-cb38-4310-a846-7d90445e913b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:41.606981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:41.666971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f8cdd-cb38-4310-a846-7d90445e913b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:41.704071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e60ff-2f1a-40a7-9b4d-9b169f72535d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:41.704702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e60ff-2f1a-40a7-9b4d-9b169f72535d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:41.704984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e60ff-2f1a-40a7-9b4d-9b169f72535d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:41.705260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e60ff-2f1a-40a7-9b4d-9b169f72535d None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:41.705979 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e60ff-2f1a-40a7-9b4d-9b169f72535d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:41.706291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e60ff-2f1a-40a7-9b4d-9b169f72535d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:41.707727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e60ff-2f1a-40a7-9b4d-9b169f72535d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:41.718201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 499/1005] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Sep 16 07:47:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:41.720837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7de82c-0e86-47c0-81a3-08c19e244e50 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:41.828629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7de82c-0e86-47c0-81a3-08c19e244e50 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:41.862407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6df04-787e-45a7-b7c0-31c2c6c5643a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:41.862971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6df04-787e-45a7-b7c0-31c2c6c5643a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:41.863252 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6df04-787e-45a7-b7c0-31c2c6c5643a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:41.863528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6df04-787e-45a7-b7c0-31c2c6c5643a None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:41.864152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6df04-787e-45a7-b7c0-31c2c6c5643a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:41.864459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6df04-787e-45a7-b7c0-31c2c6c5643a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:41.865820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6df04-787e-45a7-b7c0-31c2c6c5643a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:41.870750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 507/1006] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Sep 16 07:47:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:41.876348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e60ff-2f1a-40a7-9b4d-9b169f72535d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:41.980633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e60ff-2f1a-40a7-9b4d-9b169f72535d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:41.990030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:42.020672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3154ec5-c0cf-4296-9f15-91f1e906c63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:42.021650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3154ec5-c0cf-4296-9f15-91f1e906c63b None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:42.022085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3154ec5-c0cf-4296-9f15-91f1e906c63b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:42.022507 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3154ec5-c0cf-4296-9f15-91f1e906c63b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:42.024004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3154ec5-c0cf-4296-9f15-91f1e906c63b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:42.025726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3154ec5-c0cf-4296-9f15-91f1e906c63b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:42.114243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 500/1007] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 07:47:41 2022] POST /identity/v3/endpoints => generated 325 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:47:42.125803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb7024-99da-4b7a-b988-546e96fd66f0 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:42.126345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb7024-99da-4b7a-b988-546e96fd66f0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:42.126345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb7024-99da-4b7a-b988-546e96fd66f0 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:42.132988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 508/1008] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:42.134437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa756dd7-4ffa-4110-9c36-8d1d7d212d98 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:42.134753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa756dd7-4ffa-4110-9c36-8d1d7d212d98 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:42.135082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa756dd7-4ffa-4110-9c36-8d1d7d212d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:42.155256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fa756dd7-4ffa-4110-9c36-8d1d7d212d98 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:42.156514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fa756dd7-4ffa-4110-9c36-8d1d7d212d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:42.235702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 501/1009] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:42 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:42.505458 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb7024-99da-4b7a-b988-546e96fd66f0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:42.553114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:42.608391 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb7024-99da-4b7a-b988-546e96fd66f0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:42.643163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b19406-b660-4d86-8822-ed199fad8e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:42.643777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19406-b660-4d86-8822-ed199fad8e28 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:42.644052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19406-b660-4d86-8822-ed199fad8e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:42.644320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19406-b660-4d86-8822-ed199fad8e28 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:42.644975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b19406-b660-4d86-8822-ed199fad8e28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:42.646317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b19406-b660-4d86-8822-ed199fad8e28 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:42.754198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 509/1010] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:47:42 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:47:43.735849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab11697-85aa-4f71-b43b-66d7dd9d6599 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:43.736369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab11697-85aa-4f71-b43b-66d7dd9d6599 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:43.736448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab11697-85aa-4f71-b43b-66d7dd9d6599 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:43.738037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 502/1011] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:43.743913 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43a1e2-80e3-41cd-b4b6-b728e19df35a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:43.744315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43a1e2-80e3-41cd-b4b6-b728e19df35a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:43.744489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43a1e2-80e3-41cd-b4b6-b728e19df35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:43.762958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6d43a1e2-80e3-41cd-b4b6-b728e19df35a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:43.764162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6d43a1e2-80e3-41cd-b4b6-b728e19df35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:43.838870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 510/1012] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:43 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:44.110454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab11697-85aa-4f71-b43b-66d7dd9d6599 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:44.209164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:44.217011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab11697-85aa-4f71-b43b-66d7dd9d6599 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:44.250750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad1a1f-e7f7-4d2f-997a-ad7fecdd368d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:44.251384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1a1f-e7f7-4d2f-997a-ad7fecdd368d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:44.251668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1a1f-e7f7-4d2f-997a-ad7fecdd368d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:44.251943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1a1f-e7f7-4d2f-997a-ad7fecdd368d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:44.252529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad1a1f-e7f7-4d2f-997a-ad7fecdd368d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:44.253909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad1a1f-e7f7-4d2f-997a-ad7fecdd368d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-50ad1a1f-e7f7-4d2f-997a-ad7fecdd368d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:44.263129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:44.264115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 07:47:44.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:44.266774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 503/1013] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:44.270410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43a1e2-80e3-41cd-b4b6-b728e19df35a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:44.299576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e9e58-9d15-4cba-8ce5-fb76aabbcc0e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:44.299851 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e9e58-9d15-4cba-8ce5-fb76aabbcc0e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:44.300123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e9e58-9d15-4cba-8ce5-fb76aabbcc0e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:44.301566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 504/1014] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:44.302234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:44.306820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e7d43-9415-4ad9-bae7-3ba2df71accd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:44.307100 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e7d43-9415-4ad9-bae7-3ba2df71accd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:44.307371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e7d43-9415-4ad9-bae7-3ba2df71accd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:44.329030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-de1e7d43-9415-4ad9-bae7-3ba2df71accd None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:44.330859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-de1e7d43-9415-4ad9-bae7-3ba2df71accd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:44.383498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43a1e2-80e3-41cd-b4b6-b728e19df35a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:44.407131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 505/1015] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:44 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:44.421602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5819f0-dc52-45d6-834a-2c66e415f583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:44.422215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5819f0-dc52-45d6-834a-2c66e415f583 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:44.422497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5819f0-dc52-45d6-834a-2c66e415f583 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:44.422801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5819f0-dc52-45d6-834a-2c66e415f583 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:44.423428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5819f0-dc52-45d6-834a-2c66e415f583 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:44.423731 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5819f0-dc52-45d6-834a-2c66e415f583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:44.425087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5819f0-dc52-45d6-834a-2c66e415f583 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:44.430133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 511/1016] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:44.435743 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e7d43-9415-4ad9-bae7-3ba2df71accd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:44.526059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:44.538036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e7d43-9415-4ad9-bae7-3ba2df71accd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:44.573136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd517af7-5494-40d1-9557-9f6d2a1d79f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:44.573742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd517af7-5494-40d1-9557-9f6d2a1d79f8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:44.574021 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd517af7-5494-40d1-9557-9f6d2a1d79f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:44.574295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd517af7-5494-40d1-9557-9f6d2a1d79f8 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:44.574956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd517af7-5494-40d1-9557-9f6d2a1d79f8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:44.575262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd517af7-5494-40d1-9557-9f6d2a1d79f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:44.576630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd517af7-5494-40d1-9557-9f6d2a1d79f8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:44.586575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 506/1017] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Sep 16 07:47:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:44.588063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5819f0-dc52-45d6-834a-2c66e415f583 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:44.673947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd517af7-5494-40d1-9557-9f6d2a1d79f8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:44.680523 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:44.687504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5819f0-dc52-45d6-834a-2c66e415f583 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:44.726446 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dd7a6-1e27-4eb4-949a-147c9b649cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:44.727272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dd7a6-1e27-4eb4-949a-147c9b649cb6 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:44.727553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dd7a6-1e27-4eb4-949a-147c9b649cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:44.728201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dd7a6-1e27-4eb4-949a-147c9b649cb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:44.728883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6dd7a6-1e27-4eb4-949a-147c9b649cb6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:44.730315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6dd7a6-1e27-4eb4-949a-147c9b649cb6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:44.784030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd517af7-5494-40d1-9557-9f6d2a1d79f8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:44.798854 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 512/1018] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 07:47:44 2022] POST /identity/v3/endpoints => generated 334 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:47:44.830693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f803421-8d8d-4700-ad60-5641cb99b8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:44.831385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f803421-8d8d-4700-ad60-5641cb99b8ac None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:44.831663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f803421-8d8d-4700-ad60-5641cb99b8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:44.831928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f803421-8d8d-4700-ad60-5641cb99b8ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:44.833112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f803421-8d8d-4700-ad60-5641cb99b8ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:44.834616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f803421-8d8d-4700-ad60-5641cb99b8ac None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:44.837140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 507/1019] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:44.843196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dd7a6-1e27-4eb4-949a-147c9b649cb6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:44.956429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dd7a6-1e27-4eb4-949a-147c9b649cb6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:44.975318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c0af-1781-48a9-afbb-76f362d302bd None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:44.975600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c0af-1781-48a9-afbb-76f362d302bd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:44.975923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c0af-1781-48a9-afbb-76f362d302bd None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:44.977514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 508/1020] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:44.982955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a332b54-3697-444e-9575-15c4f21ebcb6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:44.983248 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a332b54-3697-444e-9575-15c4f21ebcb6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:44.983526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a332b54-3697-444e-9575-15c4f21ebcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:44.994442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087add9c-469a-4887-89ab-08c06a5f75e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:44.995080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087add9c-469a-4887-89ab-08c06a5f75e4 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:44.995358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087add9c-469a-4887-89ab-08c06a5f75e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:44.995627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087add9c-469a-4887-89ab-08c06a5f75e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:44.996280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087add9c-469a-4887-89ab-08c06a5f75e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:44.997682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087add9c-469a-4887-89ab-08c06a5f75e4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:45.006875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1a332b54-3697-444e-9575-15c4f21ebcb6 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:45.008592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1a332b54-3697-444e-9575-15c4f21ebcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:45.020418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:45.075852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 509/1021] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:45.091465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 513/1022] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:47:44 2022] POST /identity/v3/projects => generated 311 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:47:45.354769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a332b54-3697-444e-9575-15c4f21ebcb6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:45.454254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a332b54-3697-444e-9575-15c4f21ebcb6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:45.488922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a67268-9ac5-4c6b-b19f-865d02fe27e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:45.489560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a67268-9ac5-4c6b-b19f-865d02fe27e4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:45.489839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a67268-9ac5-4c6b-b19f-865d02fe27e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:45.490113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a67268-9ac5-4c6b-b19f-865d02fe27e4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:45.490740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a67268-9ac5-4c6b-b19f-865d02fe27e4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:45.492151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a67268-9ac5-4c6b-b19f-865d02fe27e4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-17a67268-9ac5-4c6b-b19f-865d02fe27e4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:45.501088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:45.502286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:45.503303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 510/1023] 10.222.0.59 () {56 vars in 1105 bytes} [Fri Sep 16 07:47:45 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:45.508397 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087add9c-469a-4887-89ab-08c06a5f75e4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:45.511246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:45.612195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087add9c-469a-4887-89ab-08c06a5f75e4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:45.647542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66758b3e-39a0-434b-aadb-d49146ec91b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:45.648727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66758b3e-39a0-434b-aadb-d49146ec91b5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:45.649011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66758b3e-39a0-434b-aadb-d49146ec91b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:45.649287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66758b3e-39a0-434b-aadb-d49146ec91b5 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:45.649920 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66758b3e-39a0-434b-aadb-d49146ec91b5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:45.650226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66758b3e-39a0-434b-aadb-d49146ec91b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:45.651624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66758b3e-39a0-434b-aadb-d49146ec91b5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:45.660879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 514/1024] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:45.662637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a67268-9ac5-4c6b-b19f-865d02fe27e4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:45.675491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:45.775316 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a67268-9ac5-4c6b-b19f-865d02fe27e4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:45.810559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d2423-14cf-4b5f-bf36-b21b788d7058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:45.811485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2423-14cf-4b5f-bf36-b21b788d7058 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:45.811737 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2423-14cf-4b5f-bf36-b21b788d7058 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:45.812010 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2423-14cf-4b5f-bf36-b21b788d7058 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:45.812665 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d2423-14cf-4b5f-bf36-b21b788d7058 None admin] RBAC: Adding query filter params (type=image) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:45.812974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d2423-14cf-4b5f-bf36-b21b788d7058 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:45.814382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d2423-14cf-4b5f-bf36-b21b788d7058 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:45.825529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 511/1025] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:45.827078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66758b3e-39a0-434b-aadb-d49146ec91b5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:45.894151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:45.928213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66758b3e-39a0-434b-aadb-d49146ec91b5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:45.964275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146e5a6-a890-42a4-b569-47e5d15a6611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:45.964887 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146e5a6-a890-42a4-b569-47e5d15a6611 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:45.965170 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146e5a6-a890-42a4-b569-47e5d15a6611 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:45.965445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146e5a6-a890-42a4-b569-47e5d15a6611 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:45.966146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146e5a6-a890-42a4-b569-47e5d15a6611 None admin] RBAC: Adding query filter params (interface=public, service_id=6abdf9b865394c2ebbed88759c27bab1, region_id=RegionOne) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:45.966454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146e5a6-a890-42a4-b569-47e5d15a6611 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:45.967876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146e5a6-a890-42a4-b569-47e5d15a6611 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:45.977523 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 515/1026] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Sep 16 07:47:45 2022] GET /identity/v3/endpoints?service_id=6abdf9b865394c2ebbed88759c27bab1&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:45.979166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d2423-14cf-4b5f-bf36-b21b788d7058 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:46.060746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:46.085291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d2423-14cf-4b5f-bf36-b21b788d7058 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:46.118502 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ab988-33a5-4722-a7fd-c81d8b3f7c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:46.119139 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ab988-33a5-4722-a7fd-c81d8b3f7c9c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:46.119420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ab988-33a5-4722-a7fd-c81d8b3f7c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:46.119696 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ab988-33a5-4722-a7fd-c81d8b3f7c9c None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:46.120302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ab988-33a5-4722-a7fd-c81d8b3f7c9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:46.121669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ab988-33a5-4722-a7fd-c81d8b3f7c9c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:46.134769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 512/1027] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Sep 16 07:47:45 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 07:47:47.077715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e063da-6558-4b3d-bbc6-48cae5176053 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:47.078208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e063da-6558-4b3d-bbc6-48cae5176053 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:47.078208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e063da-6558-4b3d-bbc6-48cae5176053 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:47.079691 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 516/1028] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:47.084569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2d7cd-6243-45fb-a191-9fde3617c0df None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:47.084962 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2d7cd-6243-45fb-a191-9fde3617c0df None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:47.085150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2d7cd-6243-45fb-a191-9fde3617c0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:47.104352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7fe2d7cd-6243-45fb-a191-9fde3617c0df None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:47.105531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7fe2d7cd-6243-45fb-a191-9fde3617c0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:47.174351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 513/1029] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:47.275265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00513c9-d2bb-4d46-ae4f-a1dd82593cb0 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:47.275577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00513c9-d2bb-4d46-ae4f-a1dd82593cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:47.275830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00513c9-d2bb-4d46-ae4f-a1dd82593cb0 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:47.277304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 517/1030] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:47.282286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6003fb-eb06-407f-9866-0529f0fc9671 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:47.282580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6003fb-eb06-407f-9866-0529f0fc9671 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:47.282884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6003fb-eb06-407f-9866-0529f0fc9671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:47.301191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ad6003fb-eb06-407f-9866-0529f0fc9671 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:47.302374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ad6003fb-eb06-407f-9866-0529f0fc9671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:47.364940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:47.371705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 514/1031] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:47.445240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00513c9-d2bb-4d46-ae4f-a1dd82593cb0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:47.544963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00513c9-d2bb-4d46-ae4f-a1dd82593cb0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:47.573403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:47.579393 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86626389-a901-49db-be96-87614a0dccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:47.580015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86626389-a901-49db-be96-87614a0dccdb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:47.580290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86626389-a901-49db-be96-87614a0dccdb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:47.580559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86626389-a901-49db-be96-87614a0dccdb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:47.591005 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86626389-a901-49db-be96-87614a0dccdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-86626389-a901-49db-be96-87614a0dccdb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:47.596481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:47.597403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:47.598263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:47.600100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 518/1032] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Sep 16 07:47:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:47.604061 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6003fb-eb06-407f-9866-0529f0fc9671 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:47.639930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86626389-a901-49db-be96-87614a0dccdb None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:47.712467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6003fb-eb06-407f-9866-0529f0fc9671 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:47.753500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cf444-6c05-4656-b3c2-ff6f5f2b3118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:47.754113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cf444-6c05-4656-b3c2-ff6f5f2b3118 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:47.754396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cf444-6c05-4656-b3c2-ff6f5f2b3118 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:47.754670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cf444-6c05-4656-b3c2-ff6f5f2b3118 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:47.755476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cf444-6c05-4656-b3c2-ff6f5f2b3118 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:47.755898 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cf444-6c05-4656-b3c2-ff6f5f2b3118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:47.757302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cf444-6c05-4656-b3c2-ff6f5f2b3118 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:47.758447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86626389-a901-49db-be96-87614a0dccdb None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:47.770278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 515/1033] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Sep 16 07:47:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:47.771443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cf444-6c05-4656-b3c2-ff6f5f2b3118 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:47.801078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6cd49-e92d-4eba-ac78-68151b37550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:47.801718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6cd49-e92d-4eba-ac78-68151b37550d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:47.801996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6cd49-e92d-4eba-ac78-68151b37550d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:47.802263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6cd49-e92d-4eba-ac78-68151b37550d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:47.803414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6cd49-e92d-4eba-ac78-68151b37550d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:47.804875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6cd49-e92d-4eba-ac78-68151b37550d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:47.806449 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 519/1034] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:47.812171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6cd49-e92d-4eba-ac78-68151b37550d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:47.838888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:47.870827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:47.882760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cf444-6c05-4656-b3c2-ff6f5f2b3118 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:47.914615 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6cd49-e92d-4eba-ac78-68151b37550d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:47.920224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f1c64-9717-4cc0-8e6a-6859ef19c44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:47.921041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f1c64-9717-4cc0-8e6a-6859ef19c44c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:47.921472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f1c64-9717-4cc0-8e6a-6859ef19c44c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:47.922380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f1c64-9717-4cc0-8e6a-6859ef19c44c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:47.937078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f1c64-9717-4cc0-8e6a-6859ef19c44c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:47.938781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f1c64-9717-4cc0-8e6a-6859ef19c44c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:47.950161 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3404328-d1b4-471b-ad7c-729eb6a2df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:47.950791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404328-d1b4-471b-ad7c-729eb6a2df3c None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:47.951070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404328-d1b4-471b-ad7c-729eb6a2df3c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:47.951341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3404328-d1b4-471b-ad7c-729eb6a2df3c None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:47.951974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3404328-d1b4-471b-ad7c-729eb6a2df3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:47.953369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3404328-d1b4-471b-ad7c-729eb6a2df3c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-315f1c64-9717-4cc0-8e6a-6859ef19c44c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:47.954497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:47.961080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:47.962055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:47.962055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:47.962055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:47.962055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 516/1035] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:47.962055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f1c64-9717-4cc0-8e6a-6859ef19c44c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:48.056069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 520/1036] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:47:47 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:47:48.064685 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f1c64-9717-4cc0-8e6a-6859ef19c44c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:48.080523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:48.104747 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828701b-9271-4c62-8d35-32f3be3bda80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:48.105429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828701b-9271-4c62-8d35-32f3be3bda80 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:48.105646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828701b-9271-4c62-8d35-32f3be3bda80 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:48.105899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828701b-9271-4c62-8d35-32f3be3bda80 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:48.106545 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828701b-9271-4c62-8d35-32f3be3bda80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:48.106878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828701b-9271-4c62-8d35-32f3be3bda80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:48.108283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828701b-9271-4c62-8d35-32f3be3bda80 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:48.126777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 517/1037] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:48.128251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3404328-d1b4-471b-ad7c-729eb6a2df3c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:48.244186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3404328-d1b4-471b-ad7c-729eb6a2df3c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:48.260041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:48.281042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1408a-bb59-4b01-b8d7-35b755e7192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:48.281723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1408a-bb59-4b01-b8d7-35b755e7192b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:48.282006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1408a-bb59-4b01-b8d7-35b755e7192b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:48.282283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1408a-bb59-4b01-b8d7-35b755e7192b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:48.287042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c5471-acd2-4fd1-9ca8-5094da5bfc0b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:48.287331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c5471-acd2-4fd1-9ca8-5094da5bfc0b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:48.287612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c5471-acd2-4fd1-9ca8-5094da5bfc0b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:48.289205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 518/1038] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:48.290434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1408a-bb59-4b01-b8d7-35b755e7192b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:48.291929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1408a-bb59-4b01-b8d7-35b755e7192b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:48.294277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe912d-8d9a-41ce-bebf-d1db57cf17ad None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:48.294560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe912d-8d9a-41ce-bebf-d1db57cf17ad None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:48.294860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe912d-8d9a-41ce-bebf-d1db57cf17ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d0d1408a-bb59-4b01-b8d7-35b755e7192b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:48.305010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:48.306047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:48.307034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:48.307034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:48.307034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:48.307034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:48.307034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:48.307034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:48.307034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:48.307034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:48.307034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:48.307034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:48.307773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 521/1039] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Sep 16 07:47:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:48.314314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1408a-bb59-4b01-b8d7-35b755e7192b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:48.318511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2bbe912d-8d9a-41ce-bebf-d1db57cf17ad None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:48.320374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2bbe912d-8d9a-41ce-bebf-d1db57cf17ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:48.391724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 519/1040] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:48.418605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1408a-bb59-4b01-b8d7-35b755e7192b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:48.452614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36497273-4cd4-4768-95d4-aac117c9a7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:48.453274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36497273-4cd4-4768-95d4-aac117c9a7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:48.453556 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36497273-4cd4-4768-95d4-aac117c9a7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:48.453827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36497273-4cd4-4768-95d4-aac117c9a7e6 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:48.454551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36497273-4cd4-4768-95d4-aac117c9a7e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:48.454879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36497273-4cd4-4768-95d4-aac117c9a7e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:48.456299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36497273-4cd4-4768-95d4-aac117c9a7e6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:48.473278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 522/1041] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Sep 16 07:47:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:48.478864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe912d-8d9a-41ce-bebf-d1db57cf17ad None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:48.577540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe912d-8d9a-41ce-bebf-d1db57cf17ad None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:48.579037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:48.611835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3396b3c-f808-4572-bb0d-758e7a032da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:48.612447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3396b3c-f808-4572-bb0d-758e7a032da8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:48.612720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3396b3c-f808-4572-bb0d-758e7a032da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:48.612988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3396b3c-f808-4572-bb0d-758e7a032da8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:48.657182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36497273-4cd4-4768-95d4-aac117c9a7e6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:48.684049 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3396b3c-f808-4572-bb0d-758e7a032da8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:48.685662 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3396b3c-f808-4572-bb0d-758e7a032da8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:48.691224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:48.768920 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36497273-4cd4-4768-95d4-aac117c9a7e6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:48.793701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 520/1042] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:48 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:47:48.799322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3396b3c-f808-4572-bb0d-758e7a032da8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:48.801179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:48.811390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765e7e1-5cc2-46ac-9fca-526bd6311404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:48.813653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765e7e1-5cc2-46ac-9fca-526bd6311404 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:48.813653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765e7e1-5cc2-46ac-9fca-526bd6311404 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:48.813653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765e7e1-5cc2-46ac-9fca-526bd6311404 None admin] PATH_INFO: `/v3/services/image` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:48.813653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765e7e1-5cc2-46ac-9fca-526bd6311404 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:48.814503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765e7e1-5cc2-46ac-9fca-526bd6311404 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-6765e7e1-5cc2-46ac-9fca-526bd6311404 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:48.823518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:48.824535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:48.825414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 523/1043] 10.222.0.59 () {56 vars in 1105 bytes} [Fri Sep 16 07:47:48 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:48.830583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765e7e1-5cc2-46ac-9fca-526bd6311404 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:48.909797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3396b3c-f808-4572-bb0d-758e7a032da8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:48.933773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:48.940081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765e7e1-5cc2-46ac-9fca-526bd6311404 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:48.945738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410890e-5f0f-44c2-a439-f5a12612f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:48.946371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410890e-5f0f-44c2-a439-f5a12612f8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:48.946652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410890e-5f0f-44c2-a439-f5a12612f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:48.946965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410890e-5f0f-44c2-a439-f5a12612f8f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:48.958007 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410890e-5f0f-44c2-a439-f5a12612f8f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:48.959492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410890e-5f0f-44c2-a439-f5a12612f8f5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f410890e-5f0f-44c2-a439-f5a12612f8f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:48.974603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:48.976261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:48.977597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:48.977597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:48.977597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:48.977771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d183-6710-4b26-abf4-39cd465552a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:48.977771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d183-6710-4b26-abf4-39cd465552a5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:48.977925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 521/1044] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:48.978772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d183-6710-4b26-abf4-39cd465552a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:48.979161 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d183-6710-4b26-abf4-39cd465552a5 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:48.979875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884d183-6710-4b26-abf4-39cd465552a5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:48.980247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884d183-6710-4b26-abf4-39cd465552a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:48.981719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410890e-5f0f-44c2-a439-f5a12612f8f5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:48.982173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884d183-6710-4b26-abf4-39cd465552a5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:48.988539 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 524/1045] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:48.994589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d183-6710-4b26-abf4-39cd465552a5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:49.042628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:49.087685 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410890e-5f0f-44c2-a439-f5a12612f8f5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:49.103952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d183-6710-4b26-abf4-39cd465552a5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:49.124248 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f4242-e57c-4fa1-b704-21e57b7ef997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:49.125159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f4242-e57c-4fa1-b704-21e57b7ef997 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:49.125159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f4242-e57c-4fa1-b704-21e57b7ef997 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:49.125439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f4242-e57c-4fa1-b704-21e57b7ef997 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:49.126104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f4242-e57c-4fa1-b704-21e57b7ef997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:49.126350 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f4242-e57c-4fa1-b704-21e57b7ef997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:49.127762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f4242-e57c-4fa1-b704-21e57b7ef997 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:49.140042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993b976-400f-4cbb-887d-d94a70da6ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:49.140650 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993b976-400f-4cbb-887d-d94a70da6ddf None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:49.145378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993b976-400f-4cbb-887d-d94a70da6ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:49.145503 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 522/1046] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:49.145608 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993b976-400f-4cbb-887d-d94a70da6ddf None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:49.145608 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993b976-400f-4cbb-887d-d94a70da6ddf None admin] RBAC: Adding query filter params (type=image) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:49.145608 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993b976-400f-4cbb-887d-d94a70da6ddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:49.145608 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993b976-400f-4cbb-887d-d94a70da6ddf None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:49.146377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f4242-e57c-4fa1-b704-21e57b7ef997 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:49.150287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 525/1047] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:49.155743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993b976-400f-4cbb-887d-d94a70da6ddf None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:49.181471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:49.246136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f4242-e57c-4fa1-b704-21e57b7ef997 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:49.259874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993b976-400f-4cbb-887d-d94a70da6ddf None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:49.260329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:49.280395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f2139-683a-4074-b9bc-82b9a7585ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:49.281030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f2139-683a-4074-b9bc-82b9a7585ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:49.281444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f2139-683a-4074-b9bc-82b9a7585ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:49.281717 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f2139-683a-4074-b9bc-82b9a7585ad3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:49.289121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f2139-683a-4074-b9bc-82b9a7585ad3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:49.290941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f2139-683a-4074-b9bc-82b9a7585ad3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:49.295492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27e3e0-727a-4743-9f23-aebb6542b055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:49.296126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27e3e0-727a-4743-9f23-aebb6542b055 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:49.296409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27e3e0-727a-4743-9f23-aebb6542b055 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:49.296684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27e3e0-727a-4743-9f23-aebb6542b055 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:49.297342 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27e3e0-727a-4743-9f23-aebb6542b055 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:49.298697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27e3e0-727a-4743-9f23-aebb6542b055 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f03f2139-683a-4074-b9bc-82b9a7585ad3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:49.302179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:49.303083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:49.304673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:49.304673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:49.304673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:49.304673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:49.304673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:49.304673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:49.304673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:49.304673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:49.304673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 523/1048] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:49.309629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f2139-683a-4074-b9bc-82b9a7585ad3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:49.375172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 526/1049] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 07:47:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 07:47:49.422423 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f2139-683a-4074-b9bc-82b9a7585ad3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:49.467152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a49f3-c04a-4d2d-9f20-fa7cc1c67265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:49.468087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a49f3-c04a-4d2d-9f20-fa7cc1c67265 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:49.468513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a49f3-c04a-4d2d-9f20-fa7cc1c67265 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:49.468926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a49f3-c04a-4d2d-9f20-fa7cc1c67265 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:49.469763 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a49f3-c04a-4d2d-9f20-fa7cc1c67265 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:49.470213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a49f3-c04a-4d2d-9f20-fa7cc1c67265 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:49.471931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a49f3-c04a-4d2d-9f20-fa7cc1c67265 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:49.485590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 524/1050] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:49.492123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27e3e0-727a-4743-9f23-aebb6542b055 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:49.579386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:49.601730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27e3e0-727a-4743-9f23-aebb6542b055 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:49.635777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ef61b-eebd-496f-9abe-62d0c2a5cf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:49.636309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef61b-eebd-496f-9abe-62d0c2a5cf20 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:49.636587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef61b-eebd-496f-9abe-62d0c2a5cf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:49.636857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ef61b-eebd-496f-9abe-62d0c2a5cf20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:49.637628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ef61b-eebd-496f-9abe-62d0c2a5cf20 None admin] RBAC: Adding query filter params (role.id=cd21a152b61c4235ad578e9b7a40d857, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=e32b5bf1a2524f1cbc432f38195e4c08) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:49.637937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ef61b-eebd-496f-9abe-62d0c2a5cf20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:49.639468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ef61b-eebd-496f-9abe-62d0c2a5cf20 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:49.645600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 527/1051] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:49 2022] GET /identity/v3/role_assignments?user.id=e32b5bf1a2524f1cbc432f38195e4c08&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=cd21a152b61c4235ad578e9b7a40d857 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:50.279557 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132dbda-65e1-4557-91b1-69e65f8ef09a None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:50.280083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132dbda-65e1-4557-91b1-69e65f8ef09a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:50.280268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132dbda-65e1-4557-91b1-69e65f8ef09a None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:50.281603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 525/1052] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:50.286847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ac461-ac17-49a1-9575-c402da012e37 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:50.287341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ac461-ac17-49a1-9575-c402da012e37 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:50.287426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ac461-ac17-49a1-9575-c402da012e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:50.307752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d52ac461-ac17-49a1-9575-c402da012e37 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:50.308966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d52ac461-ac17-49a1-9575-c402da012e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:50.378336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 528/1053] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:50.644500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132dbda-65e1-4557-91b1-69e65f8ef09a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:50.676183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:50.751699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132dbda-65e1-4557-91b1-69e65f8ef09a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:50.785783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da0964-4a50-4d78-8623-41448ea01664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:50.786409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da0964-4a50-4d78-8623-41448ea01664 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:50.786710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da0964-4a50-4d78-8623-41448ea01664 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:50.787009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da0964-4a50-4d78-8623-41448ea01664 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:50.798137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da0964-4a50-4d78-8623-41448ea01664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a5da0964-4a50-4d78-8623-41448ea01664 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:50.802927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:50.803928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:50.807676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 526/1054] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:50.810037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ac461-ac17-49a1-9575-c402da012e37 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:50.887249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:50.912575 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ac461-ac17-49a1-9575-c402da012e37 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:50.947156 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ea2ca-ece8-4243-bdea-8b4262634161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:50.947766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea2ca-ece8-4243-bdea-8b4262634161 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:50.948045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea2ca-ece8-4243-bdea-8b4262634161 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:50.948317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea2ca-ece8-4243-bdea-8b4262634161 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:50.948973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ea2ca-ece8-4243-bdea-8b4262634161 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:50.949276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ea2ca-ece8-4243-bdea-8b4262634161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:50.950625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ea2ca-ece8-4243-bdea-8b4262634161 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:50.964509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 529/1055] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:47:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:50.965701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da0964-4a50-4d78-8623-41448ea01664 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:51.055079 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:51.065218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da0964-4a50-4d78-8623-41448ea01664 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:51.099032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afe14c-e9fb-409a-ac8d-5b1511446cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:51.099671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afe14c-e9fb-409a-ac8d-5b1511446cdc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:51.099949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afe14c-e9fb-409a-ac8d-5b1511446cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:51.100219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afe14c-e9fb-409a-ac8d-5b1511446cdc None admin] PATH_INFO: `/v3/users/e366393253b8407bb4305ba2f078d3dc` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:51.114487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afe14c-e9fb-409a-ac8d-5b1511446cdc None admin] RBAC: Authorizing `identity:get_user(user_id=e366393253b8407bb4305ba2f078d3dc)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:51.115987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afe14c-e9fb-409a-ac8d-5b1511446cdc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:51.117776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 527/1056] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:47:50 2022] GET /identity/v3/users/e366393253b8407bb4305ba2f078d3dc => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:51.123376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ea2ca-ece8-4243-bdea-8b4262634161 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:51.224298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ea2ca-ece8-4243-bdea-8b4262634161 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:51.258130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbf2b4-2caf-4bdf-b9f4-0bb8c1e36e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:51.258762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbf2b4-2caf-4bdf-b9f4-0bb8c1e36e4c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:51.259045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbf2b4-2caf-4bdf-b9f4-0bb8c1e36e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:51.259318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbf2b4-2caf-4bdf-b9f4-0bb8c1e36e4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:51.262657 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:51.327570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbf2b4-2caf-4bdf-b9f4-0bb8c1e36e4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:51.329008 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbf2b4-2caf-4bdf-b9f4-0bb8c1e36e4c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:51.426063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 530/1057] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:47:51.432330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afe14c-e9fb-409a-ac8d-5b1511446cdc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:51.536267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afe14c-e9fb-409a-ac8d-5b1511446cdc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:51.570438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42733c83-4167-4a3a-876f-7fdfd2da53d4 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:51.570779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42733c83-4167-4a3a-876f-7fdfd2da53d4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:51.571069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42733c83-4167-4a3a-876f-7fdfd2da53d4 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:51.571696 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cb4e1-1014-42a0-85bc-cce359b6e347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:51.572425 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 531/1058] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:51.572683 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cb4e1-1014-42a0-85bc-cce359b6e347 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:51.572969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cb4e1-1014-42a0-85bc-cce359b6e347 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:51.573237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cb4e1-1014-42a0-85bc-cce359b6e347 None admin] PATH_INFO: `/v3/projects/9c69cf3369a8454e8f0675af5c059747` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:51.574365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625cb4e1-1014-42a0-85bc-cce359b6e347 None admin] RBAC: Authorizing `identity:get_project(project_id=9c69cf3369a8454e8f0675af5c059747)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:51.575836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625cb4e1-1014-42a0-85bc-cce359b6e347 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:51.577424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73b3d3-183f-4296-9788-a566b0f91274 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:51.577709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73b3d3-183f-4296-9788-a566b0f91274 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:51.578041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73b3d3-183f-4296-9788-a566b0f91274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:51.582454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 528/1059] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:47:51 2022] GET /identity/v3/projects/9c69cf3369a8454e8f0675af5c059747 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:51.583164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cb4e1-1014-42a0-85bc-cce359b6e347 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:51.599016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ac73b3d3-183f-4296-9788-a566b0f91274 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:51.600228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ac73b3d3-183f-4296-9788-a566b0f91274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:51.601525 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:51.631524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:51.672683 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 532/1060] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:51.700577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cb4e1-1014-42a0-85bc-cce359b6e347 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:51.739395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8695129-a5ca-4b2f-a044-e8c1c32171f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:51.740071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8695129-a5ca-4b2f-a044-e8c1c32171f6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:51.740322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8695129-a5ca-4b2f-a044-e8c1c32171f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:51.740591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8695129-a5ca-4b2f-a044-e8c1c32171f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:51.741866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8695129-a5ca-4b2f-a044-e8c1c32171f6 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=9c69cf3369a8454e8f0675af5c059747, user.id=e366393253b8407bb4305ba2f078d3dc) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:51.742171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8695129-a5ca-4b2f-a044-e8c1c32171f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:51.743629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8695129-a5ca-4b2f-a044-e8c1c32171f6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:51.758996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 529/1061] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:51 2022] GET /identity/v3/role_assignments?user.id=e366393253b8407bb4305ba2f078d3dc&scope.project.id=9c69cf3369a8454e8f0675af5c059747&role.id=62eb3e525077429aacc0ada63300802e => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:51.781664 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbb7bd-c5c7-4e9b-8113-3fc428722bdb None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:51.781951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbb7bd-c5c7-4e9b-8113-3fc428722bdb None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:51.782224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbb7bd-c5c7-4e9b-8113-3fc428722bdb None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:51.783791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 533/1062] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:51.789735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc98184-2b04-41e1-87f0-a325a7a9de2e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:51.789922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc98184-2b04-41e1-87f0-a325a7a9de2e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:51.790190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc98184-2b04-41e1-87f0-a325a7a9de2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:51.809615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-cdc98184-2b04-41e1-87f0-a325a7a9de2e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:51.811135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cdc98184-2b04-41e1-87f0-a325a7a9de2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:51.884527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 530/1063] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:47:51.943213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbb7bd-c5c7-4e9b-8113-3fc428722bdb None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:52.046702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbb7bd-c5c7-4e9b-8113-3fc428722bdb None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:52.081570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b0bf7-cf44-4929-ad1d-3e29fdd53b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:52.082188 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0bf7-cf44-4929-ad1d-3e29fdd53b39 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:52.082470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0bf7-cf44-4929-ad1d-3e29fdd53b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:52.084118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b0bf7-cf44-4929-ad1d-3e29fdd53b39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:52.117187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:52.152027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b0bf7-cf44-4929-ad1d-3e29fdd53b39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:52.153699 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b0bf7-cf44-4929-ad1d-3e29fdd53b39 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:52.154704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc98184-2b04-41e1-87f0-a325a7a9de2e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:52.165941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:52.258152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc98184-2b04-41e1-87f0-a325a7a9de2e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:52.258352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 534/1064] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:47:52.263743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b0bf7-cf44-4929-ad1d-3e29fdd53b39 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:52.292243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7c36a-2a94-4d77-8a1d-de559cc9df99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:52.292875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7c36a-2a94-4d77-8a1d-de559cc9df99 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:52.293151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7c36a-2a94-4d77-8a1d-de559cc9df99 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:52.293420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7c36a-2a94-4d77-8a1d-de559cc9df99 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:52.303653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7c36a-2a94-4d77-8a1d-de559cc9df99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-42a7c36a-2a94-4d77-8a1d-de559cc9df99 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:52.308831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:52.309785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:52.310628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 531/1065] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Sep 16 07:47:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:52.316031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7c36a-2a94-4d77-8a1d-de559cc9df99 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:52.332601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:52.368772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b0bf7-cf44-4929-ad1d-3e29fdd53b39 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:52.386934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:52.404243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ab109-8641-4106-9efc-fa822cdbc047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:52.404879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ab109-8641-4106-9efc-fa822cdbc047 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:52.405153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ab109-8641-4106-9efc-fa822cdbc047 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:52.405421 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ab109-8641-4106-9efc-fa822cdbc047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:52.416314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ab109-8641-4106-9efc-fa822cdbc047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:52.417746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ab109-8641-4106-9efc-fa822cdbc047 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:52.419194 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7c36a-2a94-4d77-8a1d-de559cc9df99 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-341ab109-8641-4106-9efc-fa822cdbc047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:52.433355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:52.434355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:52.435306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:52.435306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:52.435306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:52.435306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 535/1066] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:52.440560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ab109-8641-4106-9efc-fa822cdbc047 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:52.458790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44072716-c920-4eb1-9a2c-b72affd9f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:52.459493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44072716-c920-4eb1-9a2c-b72affd9f1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:52.459832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44072716-c920-4eb1-9a2c-b72affd9f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:52.460129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44072716-c920-4eb1-9a2c-b72affd9f1d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:52.461088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44072716-c920-4eb1-9a2c-b72affd9f1d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:52.461396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44072716-c920-4eb1-9a2c-b72affd9f1d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:52.462791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44072716-c920-4eb1-9a2c-b72affd9f1d4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:52.472583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 532/1067] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Sep 16 07:47:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:52.478566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44072716-c920-4eb1-9a2c-b72affd9f1d4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:52.550811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ab109-8641-4106-9efc-fa822cdbc047 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:52.569109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:52.582506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44072716-c920-4eb1-9a2c-b72affd9f1d4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:52.585954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9dd2b-5d34-423c-8977-5495527177f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:52.586560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9dd2b-5d34-423c-8977-5495527177f3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:52.594964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9dd2b-5d34-423c-8977-5495527177f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:52.594964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9dd2b-5d34-423c-8977-5495527177f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:52.594964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9dd2b-5d34-423c-8977-5495527177f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:52.594964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9dd2b-5d34-423c-8977-5495527177f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:52.594964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9dd2b-5d34-423c-8977-5495527177f3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:52.603566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 536/1068] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:52.618724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:52.624918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4607ef-096a-44d9-94e4-cb33f263a815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:52.625586 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4607ef-096a-44d9-94e4-cb33f263a815 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:52.625868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4607ef-096a-44d9-94e4-cb33f263a815 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:52.626142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4607ef-096a-44d9-94e4-cb33f263a815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:52.638415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4607ef-096a-44d9-94e4-cb33f263a815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:52.639974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4607ef-096a-44d9-94e4-cb33f263a815 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8e4607ef-096a-44d9-94e4-cb33f263a815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:52.657243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:52.658309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:52.659278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:52.659278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:52.659278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:52.659278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 533/1069] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:52.665001 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9dd2b-5d34-423c-8977-5495527177f3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:52.783208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9dd2b-5d34-423c-8977-5495527177f3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:52.819733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546ef98-e77a-4270-a546-be9eeb7f5a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:52.820269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546ef98-e77a-4270-a546-be9eeb7f5a7a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:52.820559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546ef98-e77a-4270-a546-be9eeb7f5a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:52.820837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546ef98-e77a-4270-a546-be9eeb7f5a7a None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:52.821482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546ef98-e77a-4270-a546-be9eeb7f5a7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:52.821785 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546ef98-e77a-4270-a546-be9eeb7f5a7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:52.823182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546ef98-e77a-4270-a546-be9eeb7f5a7a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:52.836021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 537/1070] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:52.841552 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4607ef-096a-44d9-94e4-cb33f263a815 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:52.948995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4607ef-096a-44d9-94e4-cb33f263a815 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:52.982038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90e631-2be6-43a1-a8ae-ed3748e9ffc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:52.982660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e631-2be6-43a1-a8ae-ed3748e9ffc6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:52.982968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e631-2be6-43a1-a8ae-ed3748e9ffc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:52.983238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90e631-2be6-43a1-a8ae-ed3748e9ffc6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:52.990519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90e631-2be6-43a1-a8ae-ed3748e9ffc6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:52.991984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90e631-2be6-43a1-a8ae-ed3748e9ffc6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9b90e631-2be6-43a1-a8ae-ed3748e9ffc6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:53.002770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:53.003779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 534/1071] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Sep 16 07:47:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:53.009654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546ef98-e77a-4270-a546-be9eeb7f5a7a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:53.049183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:53.107304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546ef98-e77a-4270-a546-be9eeb7f5a7a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:53.139373 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad11dbb-929d-4530-bcfa-d6767981dfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:53.139967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad11dbb-929d-4530-bcfa-d6767981dfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:53.140243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad11dbb-929d-4530-bcfa-d6767981dfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:53.140513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad11dbb-929d-4530-bcfa-d6767981dfa6 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:53.141223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad11dbb-929d-4530-bcfa-d6767981dfa6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:53.141524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad11dbb-929d-4530-bcfa-d6767981dfa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:53.142969 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad11dbb-929d-4530-bcfa-d6767981dfa6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:53.160804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 538/1072] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Sep 16 07:47:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:53.166430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90e631-2be6-43a1-a8ae-ed3748e9ffc6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:53.211150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:53.263158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90e631-2be6-43a1-a8ae-ed3748e9ffc6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:53.295486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf1a25-d671-4972-8c87-99996e0f1920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:53.296110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf1a25-d671-4972-8c87-99996e0f1920 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:53.296387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf1a25-d671-4972-8c87-99996e0f1920 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:53.296669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf1a25-d671-4972-8c87-99996e0f1920 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:53.306716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf1a25-d671-4972-8c87-99996e0f1920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:53.308152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf1a25-d671-4972-8c87-99996e0f1920 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3ccf1a25-d671-4972-8c87-99996e0f1920 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:53.322493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:53.323642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:53.324548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:53.324548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:53.324548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:53.324548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 535/1073] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:53.329388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad11dbb-929d-4530-bcfa-d6767981dfa6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:53.419153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:53.424668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad11dbb-929d-4530-bcfa-d6767981dfa6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:53.458951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e42a-f5a2-4299-a668-1d2f78f3ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:53.459561 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e42a-f5a2-4299-a668-1d2f78f3ff1e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:53.459841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e42a-f5a2-4299-a668-1d2f78f3ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:53.460112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82e42a-f5a2-4299-a668-1d2f78f3ff1e None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:53.460750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82e42a-f5a2-4299-a668-1d2f78f3ff1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:53.461055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82e42a-f5a2-4299-a668-1d2f78f3ff1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:53.462411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82e42a-f5a2-4299-a668-1d2f78f3ff1e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:53.473462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 539/1074] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:53.479079 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf1a25-d671-4972-8c87-99996e0f1920 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:53.573850 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:53.579523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf1a25-d671-4972-8c87-99996e0f1920 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:53.612175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3298f-f2dd-4e71-ab46-122f3feecc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:53.612809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3298f-f2dd-4e71-ab46-122f3feecc1f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:53.613087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3298f-f2dd-4e71-ab46-122f3feecc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:53.613359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3298f-f2dd-4e71-ab46-122f3feecc1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:53.620637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3298f-f2dd-4e71-ab46-122f3feecc1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:53.622081 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3298f-f2dd-4e71-ab46-122f3feecc1f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-89a3298f-f2dd-4e71-ab46-122f3feecc1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:53.633881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:53.634827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:53.635734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:53.635734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:53.635734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:53.635734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:53.635734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:53.635734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:53.635734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:53.635734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:53.635734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 536/1075] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:53.640888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e42a-f5a2-4299-a668-1d2f78f3ff1e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:53.742176 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82e42a-f5a2-4299-a668-1d2f78f3ff1e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:53.777269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c91809-2332-4a75-93c6-935b99f176a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:53.777901 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c91809-2332-4a75-93c6-935b99f176a8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:53.778180 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c91809-2332-4a75-93c6-935b99f176a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:53.778454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c91809-2332-4a75-93c6-935b99f176a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:53.779268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c91809-2332-4a75-93c6-935b99f176a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:53.779581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c91809-2332-4a75-93c6-935b99f176a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:53.781059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c91809-2332-4a75-93c6-935b99f176a8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:53.794998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 540/1076] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:53.801009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3298f-f2dd-4e71-ab46-122f3feecc1f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:53.882441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b7d6a-ee1b-4cd3-957a-7c3fff1a16bf None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:53.882856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b7d6a-ee1b-4cd3-957a-7c3fff1a16bf None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:53.883140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b7d6a-ee1b-4cd3-957a-7c3fff1a16bf None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:53.884480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 541/1077] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:53.889400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7494c17-a9c4-41ef-b0cd-d800134d1661 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:53.889746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7494c17-a9c4-41ef-b0cd-d800134d1661 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:53.890026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7494c17-a9c4-41ef-b0cd-d800134d1661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:53.900653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3298f-f2dd-4e71-ab46-122f3feecc1f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:53.907586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c7494c17-a9c4-41ef-b0cd-d800134d1661 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:53.908780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c7494c17-a9c4-41ef-b0cd-d800134d1661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:53.920083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:53.933812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398f927-ae54-44bf-8a85-243fd0e97888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:53.936577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398f927-ae54-44bf-8a85-243fd0e97888 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:53.936577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398f927-ae54-44bf-8a85-243fd0e97888 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:53.936577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398f927-ae54-44bf-8a85-243fd0e97888 None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/e32b5bf1a2524f1cbc432f38195e4c08/roles/cd21a152b61c4235ad578e9b7a40d857` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:53.937603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398f927-ae54-44bf-8a85-243fd0e97888 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=e32b5bf1a2524f1cbc432f38195e4c08, role_id=cd21a152b61c4235ad578e9b7a40d857)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:53.939055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398f927-ae54-44bf-8a85-243fd0e97888 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:53.974249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 542/1078] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:54.021051 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 537/1079] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:47:53 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/e32b5bf1a2524f1cbc432f38195e4c08/roles/cd21a152b61c4235ad578e9b7a40d857 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 07:47:54.241468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7494c17-a9c4-41ef-b0cd-d800134d1661 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:54.338717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7494c17-a9c4-41ef-b0cd-d800134d1661 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:54.371911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af2fe9-2e4d-48b7-bec7-897054ca1784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:54.372544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af2fe9-2e4d-48b7-bec7-897054ca1784 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:54.372827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af2fe9-2e4d-48b7-bec7-897054ca1784 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:54.373103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af2fe9-2e4d-48b7-bec7-897054ca1784 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:54.382961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af2fe9-2e4d-48b7-bec7-897054ca1784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-55af2fe9-2e4d-48b7-bec7-897054ca1784 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:54.387760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:54.388851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:54.390004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 543/1080] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:54.394829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398f927-ae54-44bf-8a85-243fd0e97888 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:54.414036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:54.495236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398f927-ae54-44bf-8a85-243fd0e97888 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:54.527956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306323d-b0f9-4b05-959b-e7c527e727e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:54.528566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306323d-b0f9-4b05-959b-e7c527e727e0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:54.528847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306323d-b0f9-4b05-959b-e7c527e727e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:54.529122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306323d-b0f9-4b05-959b-e7c527e727e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:54.529784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306323d-b0f9-4b05-959b-e7c527e727e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:54.530089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306323d-b0f9-4b05-959b-e7c527e727e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:54.531506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306323d-b0f9-4b05-959b-e7c527e727e0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:54.540716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 538/1081] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:47:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:54.546334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af2fe9-2e4d-48b7-bec7-897054ca1784 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:54.555982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:54.645738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af2fe9-2e4d-48b7-bec7-897054ca1784 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:54.680357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94995c-d32a-491e-9c25-250d94ca313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:54.681032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94995c-d32a-491e-9c25-250d94ca313e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:54.681319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94995c-d32a-491e-9c25-250d94ca313e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:54.681593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94995c-d32a-491e-9c25-250d94ca313e None admin] PATH_INFO: `/v3/users/e366393253b8407bb4305ba2f078d3dc` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:54.682745 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94995c-d32a-491e-9c25-250d94ca313e None admin] RBAC: Authorizing `identity:get_user(user_id=e366393253b8407bb4305ba2f078d3dc)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:54.684179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94995c-d32a-491e-9c25-250d94ca313e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:54.686154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 544/1082] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:47:54 2022] GET /identity/v3/users/e366393253b8407bb4305ba2f078d3dc => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:54.691844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306323d-b0f9-4b05-959b-e7c527e727e0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:54.775660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:54.796302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306323d-b0f9-4b05-959b-e7c527e727e0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:54.829809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd6b22-ef1b-4934-8d39-c923fcae843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:54.830438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd6b22-ef1b-4934-8d39-c923fcae843f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:54.830749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd6b22-ef1b-4934-8d39-c923fcae843f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:54.831022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd6b22-ef1b-4934-8d39-c923fcae843f None admin] PATH_INFO: `/v3/projects/9c69cf3369a8454e8f0675af5c059747` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:54.832130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd6b22-ef1b-4934-8d39-c923fcae843f None admin] RBAC: Authorizing `identity:get_project(project_id=9c69cf3369a8454e8f0675af5c059747)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:54.833552 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd6b22-ef1b-4934-8d39-c923fcae843f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:54.836074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 539/1083] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:47:54 2022] GET /identity/v3/projects/9c69cf3369a8454e8f0675af5c059747 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:54.842094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94995c-d32a-491e-9c25-250d94ca313e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:54.920343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:54.945236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94995c-d32a-491e-9c25-250d94ca313e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:54.978031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e5c4b-82ff-44ed-8138-e487aa022b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:54.978709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e5c4b-82ff-44ed-8138-e487aa022b43 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:54.978992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e5c4b-82ff-44ed-8138-e487aa022b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:54.979268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e5c4b-82ff-44ed-8138-e487aa022b43 None admin] PATH_INFO: `/v3/projects/9c69cf3369a8454e8f0675af5c059747/users/e366393253b8407bb4305ba2f078d3dc/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:54.981146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e5c4b-82ff-44ed-8138-e487aa022b43 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c69cf3369a8454e8f0675af5c059747, user_id=e366393253b8407bb4305ba2f078d3dc, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:54.982561 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e5c4b-82ff-44ed-8138-e487aa022b43 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:55.047585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 545/1084] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:47:54 2022] PUT /identity/v3/projects/9c69cf3369a8454e8f0675af5c059747/users/e366393253b8407bb4305ba2f078d3dc/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:47:56.116462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaffdf3-12b6-4a33-ade1-be921a303e8b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:56.116968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaffdf3-12b6-4a33-ade1-be921a303e8b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:56.116968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaffdf3-12b6-4a33-ade1-be921a303e8b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:56.118231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 540/1085] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:47:56.123347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff352c85-d159-4b8e-aa6f-68f8b78bba82 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:56.123751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff352c85-d159-4b8e-aa6f-68f8b78bba82 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:56.124007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff352c85-d159-4b8e-aa6f-68f8b78bba82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:56.142348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ff352c85-d159-4b8e-aa6f-68f8b78bba82 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:56.143573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ff352c85-d159-4b8e-aa6f-68f8b78bba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:56.208632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 546/1086] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:56.476932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaffdf3-12b6-4a33-ade1-be921a303e8b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:56.575839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaffdf3-12b6-4a33-ade1-be921a303e8b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:56.607631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e685a-6859-42ce-8ef6-3f5255e46915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:56.608277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e685a-6859-42ce-8ef6-3f5255e46915 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:56.608552 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e685a-6859-42ce-8ef6-3f5255e46915 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:56.608821 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e685a-6859-42ce-8ef6-3f5255e46915 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:56.618979 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e685a-6859-42ce-8ef6-3f5255e46915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1b7e685a-6859-42ce-8ef6-3f5255e46915 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:56.623645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:56.624572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:56.625493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 541/1087] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Sep 16 07:47:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:56.630710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff352c85-d159-4b8e-aa6f-68f8b78bba82 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:56.689738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:56.726129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff352c85-d159-4b8e-aa6f-68f8b78bba82 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:56.761080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f71752-a7a1-4dd0-b0b5-abc35c6c9ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:56.761683 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f71752-a7a1-4dd0-b0b5-abc35c6c9ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:56.761958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f71752-a7a1-4dd0-b0b5-abc35c6c9ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:56.762226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f71752-a7a1-4dd0-b0b5-abc35c6c9ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:56.763036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f71752-a7a1-4dd0-b0b5-abc35c6c9ac3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:56.763613 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f71752-a7a1-4dd0-b0b5-abc35c6c9ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:56.765081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f71752-a7a1-4dd0-b0b5-abc35c6c9ac3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:56.773380 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 547/1088] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Sep 16 07:47:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:56.779172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e685a-6859-42ce-8ef6-3f5255e46915 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:56.789070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:56.878775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e685a-6859-42ce-8ef6-3f5255e46915 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:56.911973 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2fae1-3619-4dbb-8a0c-a7826dacc463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:56.912606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2fae1-3619-4dbb-8a0c-a7826dacc463 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:56.913418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2fae1-3619-4dbb-8a0c-a7826dacc463 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:56.913698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2fae1-3619-4dbb-8a0c-a7826dacc463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:56.923731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2fae1-3619-4dbb-8a0c-a7826dacc463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:56.925175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2fae1-3619-4dbb-8a0c-a7826dacc463 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-cac2fae1-3619-4dbb-8a0c-a7826dacc463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:56.941101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:56.942117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:56.943075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:56.943075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:56.943075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:56.943075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 542/1089] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:56.948045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f71752-a7a1-4dd0-b0b5-abc35c6c9ac3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:57.052570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f71752-a7a1-4dd0-b0b5-abc35c6c9ac3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:57.079235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:57.087716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85beca-d600-429b-92d0-1254205e4b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:57.088319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85beca-d600-429b-92d0-1254205e4b11 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:57.088594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85beca-d600-429b-92d0-1254205e4b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:57.088861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85beca-d600-429b-92d0-1254205e4b11 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:57.089490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85beca-d600-429b-92d0-1254205e4b11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:57.095623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85beca-d600-429b-92d0-1254205e4b11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:57.095623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85beca-d600-429b-92d0-1254205e4b11 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:57.102882 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 548/1090] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:57.108483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2fae1-3619-4dbb-8a0c-a7826dacc463 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:57.153645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe89e6f-358a-43ce-b369-8b1468470e51 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:57.154191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe89e6f-358a-43ce-b369-8b1468470e51 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:57.154191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe89e6f-358a-43ce-b369-8b1468470e51 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:57.155689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 549/1091] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 07:47:57.160497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad747f-83e7-483b-a457-6f5f6ab5e64a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:57.160781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad747f-83e7-483b-a457-6f5f6ab5e64a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:57.161056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad747f-83e7-483b-a457-6f5f6ab5e64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:57.166967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:57.179430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-37ad747f-83e7-483b-a457-6f5f6ab5e64a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:47:57.180647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-37ad747f-83e7-483b-a457-6f5f6ab5e64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:47:57.208790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2fae1-3619-4dbb-8a0c-a7826dacc463 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:57.241657 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43926cb9-b34a-4911-adfb-ad7eb87d72b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:57.242286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43926cb9-b34a-4911-adfb-ad7eb87d72b7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:57.242566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43926cb9-b34a-4911-adfb-ad7eb87d72b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:57.242875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43926cb9-b34a-4911-adfb-ad7eb87d72b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:57.246293 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 550/1092] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:47:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 07:47:57.250153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43926cb9-b34a-4911-adfb-ad7eb87d72b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:57.251635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43926cb9-b34a-4911-adfb-ad7eb87d72b7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-43926cb9-b34a-4911-adfb-ad7eb87d72b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:47:57.262619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:57.263617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:57.264842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:57.264842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:57.264842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:47:57.264842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:47:57.264842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:47:57.264842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:47:57.264842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:47:57.264842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:47:57.264842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 07:47:57.264842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:57.264842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 543/1093] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Sep 16 07:47:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:57.269385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad747f-83e7-483b-a457-6f5f6ab5e64a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:57.365549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad747f-83e7-483b-a457-6f5f6ab5e64a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:57.380964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:57.398099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418529b4-59b0-487b-8223-cfd786e0e934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:57.398715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418529b4-59b0-487b-8223-cfd786e0e934 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:57.398989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418529b4-59b0-487b-8223-cfd786e0e934 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:57.399258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418529b4-59b0-487b-8223-cfd786e0e934 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:57.399967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418529b4-59b0-487b-8223-cfd786e0e934 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:57.400272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418529b4-59b0-487b-8223-cfd786e0e934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:57.401683 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418529b4-59b0-487b-8223-cfd786e0e934 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:57.418702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 551/1094] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Sep 16 07:47:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:57.424259 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43926cb9-b34a-4911-adfb-ad7eb87d72b7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:57.512369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418529b4-59b0-487b-8223-cfd786e0e934 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:57.519621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43926cb9-b34a-4911-adfb-ad7eb87d72b7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:57.536885 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:57.553568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ad03e-3afc-4b74-bd4a-ca9c6aa90aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:57.554170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ad03e-3afc-4b74-bd4a-ca9c6aa90aeb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:57.554448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ad03e-3afc-4b74-bd4a-ca9c6aa90aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:57.554755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ad03e-3afc-4b74-bd4a-ca9c6aa90aeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:57.613051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418529b4-59b0-487b-8223-cfd786e0e934 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:57.621382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ad03e-3afc-4b74-bd4a-ca9c6aa90aeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:57.622815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ad03e-3afc-4b74-bd4a-ca9c6aa90aeb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:57.645957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328ac22-915b-4e77-a984-266c21ef8158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:57.646587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328ac22-915b-4e77-a984-266c21ef8158 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:57.646897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328ac22-915b-4e77-a984-266c21ef8158 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:57.647717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328ac22-915b-4e77-a984-266c21ef8158 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:57.657526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328ac22-915b-4e77-a984-266c21ef8158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-3328ac22-915b-4e77-a984-266c21ef8158 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:47:57.662317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:57.663360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:47:57.664829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 552/1095] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:47:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:57.669309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328ac22-915b-4e77-a984-266c21ef8158 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:57.703014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:57.723563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 544/1096] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:47:57.729369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ad03e-3afc-4b74-bd4a-ca9c6aa90aeb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:57.748136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:57.771568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328ac22-915b-4e77-a984-266c21ef8158 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:57.805067 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61794415-303f-41b6-80bb-b6cfb3b79131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:57.805677 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61794415-303f-41b6-80bb-b6cfb3b79131 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:57.805961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61794415-303f-41b6-80bb-b6cfb3b79131 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:57.806236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61794415-303f-41b6-80bb-b6cfb3b79131 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:57.806929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61794415-303f-41b6-80bb-b6cfb3b79131 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:57.807239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61794415-303f-41b6-80bb-b6cfb3b79131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:57.808603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61794415-303f-41b6-80bb-b6cfb3b79131 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:57.817175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 553/1097] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:47:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:47:57.823792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61794415-303f-41b6-80bb-b6cfb3b79131 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:57.829617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ad03e-3afc-4b74-bd4a-ca9c6aa90aeb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:57.863636 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9d79c-36cf-4852-988f-b848cb2610de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:57.864263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9d79c-36cf-4852-988f-b848cb2610de None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:57.864551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9d79c-36cf-4852-988f-b848cb2610de None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:57.864824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9d79c-36cf-4852-988f-b848cb2610de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:57.874793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9d79c-36cf-4852-988f-b848cb2610de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:57.876201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9d79c-36cf-4852-988f-b848cb2610de None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-88a9d79c-36cf-4852-988f-b848cb2610de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:47:57.891138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:57.892064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:47:57.892966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:47:57.892966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:47:57.892966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:57.892966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 545/1098] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:47:57.898547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9d79c-36cf-4852-988f-b848cb2610de None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:57.916580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:57.922443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61794415-303f-41b6-80bb-b6cfb3b79131 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:57.957744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1181e-cc92-4ef3-8228-f21867360414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:57.958377 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1181e-cc92-4ef3-8228-f21867360414 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:57.958663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1181e-cc92-4ef3-8228-f21867360414 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:57.958968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1181e-cc92-4ef3-8228-f21867360414 None admin] PATH_INFO: `/v3/users/e366393253b8407bb4305ba2f078d3dc` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:57.960122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1181e-cc92-4ef3-8228-f21867360414 None admin] RBAC: Authorizing `identity:get_user(user_id=e366393253b8407bb4305ba2f078d3dc)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:57.961554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1181e-cc92-4ef3-8228-f21867360414 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:57.963323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 554/1099] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:47:57 2022] GET /identity/v3/users/e366393253b8407bb4305ba2f078d3dc => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:57.966673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:57.969153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1181e-cc92-4ef3-8228-f21867360414 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:57.997277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9d79c-36cf-4852-988f-b848cb2610de None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:58.032415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a13d4-9dbd-4fbd-93a2-1b50574839c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:58.033026 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a13d4-9dbd-4fbd-93a2-1b50574839c6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:58.033306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a13d4-9dbd-4fbd-93a2-1b50574839c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:58.033580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a13d4-9dbd-4fbd-93a2-1b50574839c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:58.034220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a13d4-9dbd-4fbd-93a2-1b50574839c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:58.034529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a13d4-9dbd-4fbd-93a2-1b50574839c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:58.035927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a13d4-9dbd-4fbd-93a2-1b50574839c6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:58.047777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 546/1100] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:58.053160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a13d4-9dbd-4fbd-93a2-1b50574839c6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:58.073771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1181e-cc92-4ef3-8228-f21867360414 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:58.107474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245105a5-31b2-43b8-84dd-713b8331b682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:58.108073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245105a5-31b2-43b8-84dd-713b8331b682 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:58.108348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245105a5-31b2-43b8-84dd-713b8331b682 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:58.108617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245105a5-31b2-43b8-84dd-713b8331b682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:58.142751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:58.155302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a13d4-9dbd-4fbd-93a2-1b50574839c6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:58.179576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245105a5-31b2-43b8-84dd-713b8331b682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:58.181051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245105a5-31b2-43b8-84dd-713b8331b682 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:58.183852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:58.189934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2495cb-c60e-45a1-94ae-8288b9c7e5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:58.190562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2495cb-c60e-45a1-94ae-8288b9c7e5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:58.190878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2495cb-c60e-45a1-94ae-8288b9c7e5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:58.191153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2495cb-c60e-45a1-94ae-8288b9c7e5c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:58.198295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2495cb-c60e-45a1-94ae-8288b9c7e5c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:58.199762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2495cb-c60e-45a1-94ae-8288b9c7e5c3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-fa2495cb-c60e-45a1-94ae-8288b9c7e5c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:47:58.211264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:47:58.212150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:47:58.213074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:47:58.213074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:47:58.213074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:47:58.213074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:47:58.213074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:47:58.213074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:47:58.213074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:47:58.213074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:47:58.213074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 547/1101] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:47:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:47:58.218038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2495cb-c60e-45a1-94ae-8288b9c7e5c3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:58.279236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 555/1102] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:47:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:47:58.285053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245105a5-31b2-43b8-84dd-713b8331b682 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:58.315972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2495cb-c60e-45a1-94ae-8288b9c7e5c3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:58.344682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:58.348897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c24906-0290-49ec-a868-5c4b3ea106d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:58.349531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c24906-0290-49ec-a868-5c4b3ea106d7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:58.349848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c24906-0290-49ec-a868-5c4b3ea106d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:58.350130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c24906-0290-49ec-a868-5c4b3ea106d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:58.355261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c24906-0290-49ec-a868-5c4b3ea106d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:58.355261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c24906-0290-49ec-a868-5c4b3ea106d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:58.355261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c24906-0290-49ec-a868-5c4b3ea106d7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:58.364311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 548/1103] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:47:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:58.370298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c24906-0290-49ec-a868-5c4b3ea106d7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:58.382028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245105a5-31b2-43b8-84dd-713b8331b682 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:58.416630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490af66-10a3-4ffc-9409-ae948b531b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:58.417195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490af66-10a3-4ffc-9409-ae948b531b8c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:58.417473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490af66-10a3-4ffc-9409-ae948b531b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:58.417746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490af66-10a3-4ffc-9409-ae948b531b8c None admin] PATH_INFO: `/v3/projects/9c69cf3369a8454e8f0675af5c059747` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:58.418396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:58.418890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490af66-10a3-4ffc-9409-ae948b531b8c None admin] RBAC: Authorizing `identity:get_project(project_id=9c69cf3369a8454e8f0675af5c059747)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:58.420676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490af66-10a3-4ffc-9409-ae948b531b8c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:58.422288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 556/1104] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:47:58 2022] GET /identity/v3/projects/9c69cf3369a8454e8f0675af5c059747 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:58.428282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490af66-10a3-4ffc-9409-ae948b531b8c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:47:58.469522 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c24906-0290-49ec-a868-5c4b3ea106d7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:58.503509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bc9a1-1f90-45ea-b7f2-2b2640529da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:58.504110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bc9a1-1f90-45ea-b7f2-2b2640529da6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:58.504392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bc9a1-1f90-45ea-b7f2-2b2640529da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:58.504664 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bc9a1-1f90-45ea-b7f2-2b2640529da6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:58.505405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bc9a1-1f90-45ea-b7f2-2b2640529da6 None admin] RBAC: Adding query filter params (role.id=cd21a152b61c4235ad578e9b7a40d857, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=e32b5bf1a2524f1cbc432f38195e4c08) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:58.505694 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bc9a1-1f90-45ea-b7f2-2b2640529da6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:58.507659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bc9a1-1f90-45ea-b7f2-2b2640529da6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:58.513217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 549/1105] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:58 2022] GET /identity/v3/role_assignments?user.id=e32b5bf1a2524f1cbc432f38195e4c08&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=cd21a152b61c4235ad578e9b7a40d857 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:47:58.527105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490af66-10a3-4ffc-9409-ae948b531b8c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:47:58.558868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:47:58.564153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f4140-2145-407c-824e-685036b47aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:47:58.564793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f4140-2145-407c-824e-685036b47aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:47:58.565076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f4140-2145-407c-824e-685036b47aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:47:58.565354 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f4140-2145-407c-824e-685036b47aa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:47:58.566134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f4140-2145-407c-824e-685036b47aa2 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=9c69cf3369a8454e8f0675af5c059747, user.id=e366393253b8407bb4305ba2f078d3dc) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:47:58.566447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f4140-2145-407c-824e-685036b47aa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:47:58.568010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f4140-2145-407c-824e-685036b47aa2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:47:58.575260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 557/1106] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:47:58 2022] GET /identity/v3/role_assignments?user.id=e366393253b8407bb4305ba2f078d3dc&scope.project.id=9c69cf3369a8454e8f0675af5c059747&role.id=62eb3e525077429aacc0ada63300802e => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:48:00.726860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268385-b13e-444f-8326-702469d36c51 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:00.727383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268385-b13e-444f-8326-702469d36c51 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:00.727383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268385-b13e-444f-8326-702469d36c51 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:00.728710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 550/1107] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:00.733948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1df06f-e339-4c37-8403-ebaf6b6dc558 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:00.734376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1df06f-e339-4c37-8403-ebaf6b6dc558 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:00.734506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1df06f-e339-4c37-8403-ebaf6b6dc558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:00.752870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ad1df06f-e339-4c37-8403-ebaf6b6dc558 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:00.754053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ad1df06f-e339-4c37-8403-ebaf6b6dc558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:00.818093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 558/1108] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:01.087564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35268385-b13e-444f-8326-702469d36c51 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:01.184739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35268385-b13e-444f-8326-702469d36c51 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:01.199147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:01.218230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bd214-40b4-4c77-bd69-efe3a983a4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:01.218870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bd214-40b4-4c77-bd69-efe3a983a4ab None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:01.219145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bd214-40b4-4c77-bd69-efe3a983a4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:01.219413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bd214-40b4-4c77-bd69-efe3a983a4ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:01.220729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bd214-40b4-4c77-bd69-efe3a983a4ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:01.222491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bd214-40b4-4c77-bd69-efe3a983a4ab None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:01.224361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 551/1109] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:48:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:01.230275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1df06f-e339-4c37-8403-ebaf6b6dc558 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:01.325640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1df06f-e339-4c37-8403-ebaf6b6dc558 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:01.350195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:01.358369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a742b1-4bae-473a-919e-2133c9b5b207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:01.360082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a742b1-4bae-473a-919e-2133c9b5b207 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:01.360082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a742b1-4bae-473a-919e-2133c9b5b207 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:01.360082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a742b1-4bae-473a-919e-2133c9b5b207 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:01.360357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a742b1-4bae-473a-919e-2133c9b5b207 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:01.361618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a742b1-4bae-473a-919e-2133c9b5b207 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:01.443617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 559/1110] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:48:01 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:48:03.606042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fb2e2-ad30-47c0-b685-03e013fca66e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:03.606594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fb2e2-ad30-47c0-b685-03e013fca66e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:03.606594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fb2e2-ad30-47c0-b685-03e013fca66e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:03.607977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 552/1111] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:03.613275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d58db-7764-4fcd-9cbc-96d1a085f0c1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:03.613713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d58db-7764-4fcd-9cbc-96d1a085f0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:03.613814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d58db-7764-4fcd-9cbc-96d1a085f0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:03.631953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0e6d58db-7764-4fcd-9cbc-96d1a085f0c1 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:03.633152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0e6d58db-7764-4fcd-9cbc-96d1a085f0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:03.697733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 560/1112] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:03.966550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fb2e2-ad30-47c0-b685-03e013fca66e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:04.065441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fb2e2-ad30-47c0-b685-03e013fca66e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:04.098649 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96b670-bd76-48f2-986e-c45eca969227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:04.099306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96b670-bd76-48f2-986e-c45eca969227 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:04.099585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96b670-bd76-48f2-986e-c45eca969227 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:04.099882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96b670-bd76-48f2-986e-c45eca969227 None admin] PATH_INFO: `/v3/roles/64e24908c05145b49cc926880ab769cf` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:04.101265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96b670-bd76-48f2-986e-c45eca969227 None admin] RBAC: Authorizing `identity:get_role(role_id=64e24908c05145b49cc926880ab769cf)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:04.102622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96b670-bd76-48f2-986e-c45eca969227 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:04.103932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 553/1113] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:03 2022] GET /identity/v3/roles/64e24908c05145b49cc926880ab769cf => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:04.109571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d58db-7764-4fcd-9cbc-96d1a085f0c1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:04.204219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d58db-7764-4fcd-9cbc-96d1a085f0c1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:04.216546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:04.236085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb6b63-dc3f-446e-96da-c2ca221be849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:04.236693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb6b63-dc3f-446e-96da-c2ca221be849 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:04.237164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb6b63-dc3f-446e-96da-c2ca221be849 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:04.237635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb6b63-dc3f-446e-96da-c2ca221be849 None admin] PATH_INFO: `/v3/users/8011a21643d44f6aa05e950bd6dd0ca0` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:04.251128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb6b63-dc3f-446e-96da-c2ca221be849 None admin] RBAC: Authorizing `identity:get_user(user_id=8011a21643d44f6aa05e950bd6dd0ca0)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:04.252633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb6b63-dc3f-446e-96da-c2ca221be849 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:04.254257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 561/1114] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:04 2022] GET /identity/v3/users/8011a21643d44f6aa05e950bd6dd0ca0 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:04.260600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96b670-bd76-48f2-986e-c45eca969227 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:04.309340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:04.356445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96b670-bd76-48f2-986e-c45eca969227 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:04.388529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73346e0-a367-4b0e-8089-584e51946506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:04.389118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73346e0-a367-4b0e-8089-584e51946506 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:04.389393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73346e0-a367-4b0e-8089-584e51946506 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:04.389660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73346e0-a367-4b0e-8089-584e51946506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:04.452981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73346e0-a367-4b0e-8089-584e51946506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:04.454491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73346e0-a367-4b0e-8089-584e51946506 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:04.501272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:04.549053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 554/1115] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:48:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:48:04.555754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb6b63-dc3f-446e-96da-c2ca221be849 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:04.659174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb6b63-dc3f-446e-96da-c2ca221be849 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:04.692941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f44bb-a11c-4188-a465-153700ed1245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:04.693564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f44bb-a11c-4188-a465-153700ed1245 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:04.693845 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f44bb-a11c-4188-a465-153700ed1245 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:04.694119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f44bb-a11c-4188-a465-153700ed1245 None admin] PATH_INFO: `/v3/projects/9c69cf3369a8454e8f0675af5c059747` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:04.695244 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f44bb-a11c-4188-a465-153700ed1245 None admin] RBAC: Authorizing `identity:get_project(project_id=9c69cf3369a8454e8f0675af5c059747)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:04.696688 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f44bb-a11c-4188-a465-153700ed1245 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:04.698296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 562/1116] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:48:04 2022] GET /identity/v3/projects/9c69cf3369a8454e8f0675af5c059747 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:04.704594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73346e0-a367-4b0e-8089-584e51946506 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:04.802883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73346e0-a367-4b0e-8089-584e51946506 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:04.836778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cbda6-5b1d-4f49-afba-a7894ecfb9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:04.837322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cbda6-5b1d-4f49-afba-a7894ecfb9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:04.837598 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cbda6-5b1d-4f49-afba-a7894ecfb9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:04.837866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cbda6-5b1d-4f49-afba-a7894ecfb9f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:04.838576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cbda6-5b1d-4f49-afba-a7894ecfb9f4 None admin] RBAC: Adding query filter params (role.id=64e24908c05145b49cc926880ab769cf, scope.project.id=9c69cf3369a8454e8f0675af5c059747, user.id=8011a21643d44f6aa05e950bd6dd0ca0) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:04.838910 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cbda6-5b1d-4f49-afba-a7894ecfb9f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:04.840331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cbda6-5b1d-4f49-afba-a7894ecfb9f4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:04.845614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 555/1117] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:48:04 2022] GET /identity/v3/role_assignments?user.id=8011a21643d44f6aa05e950bd6dd0ca0&scope.project.id=9c69cf3369a8454e8f0675af5c059747&role.id=64e24908c05145b49cc926880ab769cf => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:06.949498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4d0c4-dcca-4d21-a6f4-cd2ce0b88846 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:06.950039 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4d0c4-dcca-4d21-a6f4-cd2ce0b88846 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:06.950039 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4d0c4-dcca-4d21-a6f4-cd2ce0b88846 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:06.951395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 563/1118] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:06.956453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b78c2-9d14-47c2-9ef7-9df6b523f2f5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:06.956875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b78c2-9d14-47c2-9ef7-9df6b523f2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:06.956984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b78c2-9d14-47c2-9ef7-9df6b523f2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:06.974738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-361b78c2-9d14-47c2-9ef7-9df6b523f2f5 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:06.975913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-361b78c2-9d14-47c2-9ef7-9df6b523f2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:06.984807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:07.039495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 556/1119] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:07.306170 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4d0c4-dcca-4d21-a6f4-cd2ce0b88846 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:07.345379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:07.402868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4d0c4-dcca-4d21-a6f4-cd2ce0b88846 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:07.435672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f00c80-98ff-4839-b987-57b4cfa751fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:07.436285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f00c80-98ff-4839-b987-57b4cfa751fc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:07.436559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f00c80-98ff-4839-b987-57b4cfa751fc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:07.436828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f00c80-98ff-4839-b987-57b4cfa751fc None admin] PATH_INFO: `/v3/roles/64e24908c05145b49cc926880ab769cf` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:07.438185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f00c80-98ff-4839-b987-57b4cfa751fc None admin] RBAC: Authorizing `identity:get_role(role_id=64e24908c05145b49cc926880ab769cf)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:07.439563 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f00c80-98ff-4839-b987-57b4cfa751fc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:07.440839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 564/1120] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:07 2022] GET /identity/v3/roles/64e24908c05145b49cc926880ab769cf => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:07.446498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b78c2-9d14-47c2-9ef7-9df6b523f2f5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:07.541520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b78c2-9d14-47c2-9ef7-9df6b523f2f5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:07.574531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4f675-89cc-4a85-81c0-e67979508ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:07.575118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4f675-89cc-4a85-81c0-e67979508ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:07.575398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4f675-89cc-4a85-81c0-e67979508ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:07.575666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4f675-89cc-4a85-81c0-e67979508ad0 None admin] PATH_INFO: `/v3/users/8011a21643d44f6aa05e950bd6dd0ca0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:07.576777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4f675-89cc-4a85-81c0-e67979508ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=8011a21643d44f6aa05e950bd6dd0ca0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:07.578197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4f675-89cc-4a85-81c0-e67979508ad0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:07.579762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 557/1121] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:07 2022] GET /identity/v3/users/8011a21643d44f6aa05e950bd6dd0ca0 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:07.585308 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f00c80-98ff-4839-b987-57b4cfa751fc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:07.680350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f00c80-98ff-4839-b987-57b4cfa751fc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:07.687470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:07.712592 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db95b4-1b50-40cd-9e47-cf6c23d48c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:07.713200 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db95b4-1b50-40cd-9e47-cf6c23d48c3e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:07.713475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db95b4-1b50-40cd-9e47-cf6c23d48c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:07.722123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db95b4-1b50-40cd-9e47-cf6c23d48c3e None admin] PATH_INFO: `/v3/projects/9c69cf3369a8454e8f0675af5c059747` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:07.722123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db95b4-1b50-40cd-9e47-cf6c23d48c3e None admin] RBAC: Authorizing `identity:get_project(project_id=9c69cf3369a8454e8f0675af5c059747)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:07.722123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db95b4-1b50-40cd-9e47-cf6c23d48c3e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:07.722123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 565/1122] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:48:07 2022] GET /identity/v3/projects/9c69cf3369a8454e8f0675af5c059747 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:07.723886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4f675-89cc-4a85-81c0-e67979508ad0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:07.739241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:07.820490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4f675-89cc-4a85-81c0-e67979508ad0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:07.853284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9901cc1-21a4-4c91-9594-72cd2bf9849c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:07.853870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9901cc1-21a4-4c91-9594-72cd2bf9849c None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:07.854144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9901cc1-21a4-4c91-9594-72cd2bf9849c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:07.854412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9901cc1-21a4-4c91-9594-72cd2bf9849c None admin] PATH_INFO: `/v3/projects/9c69cf3369a8454e8f0675af5c059747/users/8011a21643d44f6aa05e950bd6dd0ca0/roles/64e24908c05145b49cc926880ab769cf` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:07.856518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9901cc1-21a4-4c91-9594-72cd2bf9849c None admin] RBAC: Authorizing `identity:create_grant(project_id=9c69cf3369a8454e8f0675af5c059747, user_id=8011a21643d44f6aa05e950bd6dd0ca0, role_id=64e24908c05145b49cc926880ab769cf)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:07.857935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9901cc1-21a4-4c91-9594-72cd2bf9849c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:08.049288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 558/1123] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:48:07 2022] PUT /identity/v3/projects/9c69cf3369a8454e8f0675af5c059747/users/8011a21643d44f6aa05e950bd6dd0ca0/roles/64e24908c05145b49cc926880ab769cf => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:48:10.144328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe718ee2-479b-4dc9-9bd5-94d826849b12 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:10.144855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe718ee2-479b-4dc9-9bd5-94d826849b12 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:10.144855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe718ee2-479b-4dc9-9bd5-94d826849b12 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:10.146129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 566/1124] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:10.151409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81dd5-6321-41e8-8a49-732e511c6300 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:10.151802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81dd5-6321-41e8-8a49-732e511c6300 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:10.151951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81dd5-6321-41e8-8a49-732e511c6300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:10.169486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d0a81dd5-6321-41e8-8a49-732e511c6300 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:10.170650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d0a81dd5-6321-41e8-8a49-732e511c6300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:10.235613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 559/1125] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:10.500075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe718ee2-479b-4dc9-9bd5-94d826849b12 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:10.597491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe718ee2-479b-4dc9-9bd5-94d826849b12 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:10.630020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fcdde-ab5c-4014-8856-79ca319f14ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:10.630631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fcdde-ab5c-4014-8856-79ca319f14ea None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:10.630958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fcdde-ab5c-4014-8856-79ca319f14ea None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:10.631227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fcdde-ab5c-4014-8856-79ca319f14ea None admin] PATH_INFO: `/v3/roles/64e24908c05145b49cc926880ab769cf` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:10.632582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fcdde-ab5c-4014-8856-79ca319f14ea None admin] RBAC: Authorizing `identity:get_role(role_id=64e24908c05145b49cc926880ab769cf)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:10.633931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fcdde-ab5c-4014-8856-79ca319f14ea None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:10.635240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 567/1126] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:10 2022] GET /identity/v3/roles/64e24908c05145b49cc926880ab769cf => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:10.640803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a81dd5-6321-41e8-8a49-732e511c6300 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:10.642965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:10.735918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a81dd5-6321-41e8-8a49-732e511c6300 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:10.768173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca70d4-5a83-4b39-9ecb-b3ae8553ae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:10.768726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca70d4-5a83-4b39-9ecb-b3ae8553ae33 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:10.769000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca70d4-5a83-4b39-9ecb-b3ae8553ae33 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:10.769268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca70d4-5a83-4b39-9ecb-b3ae8553ae33 None admin] PATH_INFO: `/v3/users/8011a21643d44f6aa05e950bd6dd0ca0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:10.770366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca70d4-5a83-4b39-9ecb-b3ae8553ae33 None admin] RBAC: Authorizing `identity:get_user(user_id=8011a21643d44f6aa05e950bd6dd0ca0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:10.772339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca70d4-5a83-4b39-9ecb-b3ae8553ae33 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:10.773905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 560/1127] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:10 2022] GET /identity/v3/users/8011a21643d44f6aa05e950bd6dd0ca0 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:10.779877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fcdde-ab5c-4014-8856-79ca319f14ea None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:10.822207 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:10.876232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fcdde-ab5c-4014-8856-79ca319f14ea None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:10.908273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e694b9-bd0d-45e5-b65e-dadb963d3b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:10.908861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e694b9-bd0d-45e5-b65e-dadb963d3b8b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:10.909137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e694b9-bd0d-45e5-b65e-dadb963d3b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:10.909406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e694b9-bd0d-45e5-b65e-dadb963d3b8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:10.972412 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e694b9-bd0d-45e5-b65e-dadb963d3b8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:10.973821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e694b9-bd0d-45e5-b65e-dadb963d3b8b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:11.012574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:11.067239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 568/1128] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:48:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:48:11.073685 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca70d4-5a83-4b39-9ecb-b3ae8553ae33 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:11.140711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:11.169867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca70d4-5a83-4b39-9ecb-b3ae8553ae33 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:11.203751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86c2e3-3362-47c1-b7f1-5d246243b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:11.204376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86c2e3-3362-47c1-b7f1-5d246243b753 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:11.204657 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86c2e3-3362-47c1-b7f1-5d246243b753 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:11.204934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86c2e3-3362-47c1-b7f1-5d246243b753 None admin] PATH_INFO: `/v3/projects/9c69cf3369a8454e8f0675af5c059747` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:11.206054 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86c2e3-3362-47c1-b7f1-5d246243b753 None admin] RBAC: Authorizing `identity:get_project(project_id=9c69cf3369a8454e8f0675af5c059747)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:11.207513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86c2e3-3362-47c1-b7f1-5d246243b753 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:11.209745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 561/1129] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:48:11 2022] GET /identity/v3/projects/9c69cf3369a8454e8f0675af5c059747 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:11.215853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e694b9-bd0d-45e5-b65e-dadb963d3b8b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:11.313435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e694b9-bd0d-45e5-b65e-dadb963d3b8b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:11.344956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46e347-c7c0-4fde-b7c0-19fd4fcf1a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:11.345490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46e347-c7c0-4fde-b7c0-19fd4fcf1a1a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:11.345766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46e347-c7c0-4fde-b7c0-19fd4fcf1a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:11.346038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46e347-c7c0-4fde-b7c0-19fd4fcf1a1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:11.346772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46e347-c7c0-4fde-b7c0-19fd4fcf1a1a None admin] RBAC: Adding query filter params (role.id=64e24908c05145b49cc926880ab769cf, scope.project.id=9c69cf3369a8454e8f0675af5c059747, user.id=8011a21643d44f6aa05e950bd6dd0ca0) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:11.347080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46e347-c7c0-4fde-b7c0-19fd4fcf1a1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:11.348490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46e347-c7c0-4fde-b7c0-19fd4fcf1a1a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:11.354496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 569/1130] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:48:11 2022] GET /identity/v3/role_assignments?user.id=8011a21643d44f6aa05e950bd6dd0ca0&scope.project.id=9c69cf3369a8454e8f0675af5c059747&role.id=64e24908c05145b49cc926880ab769cf => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:13.487061 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755a3c7-0822-45f3-95c9-73fe7e8ea972 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:13.487593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755a3c7-0822-45f3-95c9-73fe7e8ea972 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:13.487593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755a3c7-0822-45f3-95c9-73fe7e8ea972 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:13.488920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 562/1131] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:13.494193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377af0ff-2697-473b-821d-64acf807e476 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:13.494582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377af0ff-2697-473b-821d-64acf807e476 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:13.494778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377af0ff-2697-473b-821d-64acf807e476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:13.507829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:13.513148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-377af0ff-2697-473b-821d-64acf807e476 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:13.514328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-377af0ff-2697-473b-821d-64acf807e476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:13.579585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 570/1132] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:13.851070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755a3c7-0822-45f3-95c9-73fe7e8ea972 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:13.948792 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755a3c7-0822-45f3-95c9-73fe7e8ea972 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:13.980324 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af10143-30e6-4333-a778-92b9a447d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:13.980936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af10143-30e6-4333-a778-92b9a447d74e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:13.981212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af10143-30e6-4333-a778-92b9a447d74e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:13.981481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af10143-30e6-4333-a778-92b9a447d74e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:13.982573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af10143-30e6-4333-a778-92b9a447d74e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:13.984354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af10143-30e6-4333-a778-92b9a447d74e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:13.985868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 563/1133] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:48:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:13.992154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377af0ff-2697-473b-821d-64acf807e476 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:14.087153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377af0ff-2697-473b-821d-64acf807e476 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:14.120127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e699b7-5121-4f38-9a32-17bd134f2f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:14.120716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e699b7-5121-4f38-9a32-17bd134f2f95 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:14.121002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e699b7-5121-4f38-9a32-17bd134f2f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:14.121273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e699b7-5121-4f38-9a32-17bd134f2f95 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:14.121921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e699b7-5121-4f38-9a32-17bd134f2f95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:14.123363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e699b7-5121-4f38-9a32-17bd134f2f95 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:14.128491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:14.221214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 571/1134] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:48:13 2022] POST /identity/v3/projects => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:48:16.341507 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6e2c0-b352-49c1-9fbd-cd4901a4626b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:16.342039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6e2c0-b352-49c1-9fbd-cd4901a4626b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:16.342039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6e2c0-b352-49c1-9fbd-cd4901a4626b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:16.343395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 564/1135] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:16.348821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939fa0a-fb86-498d-9fb4-af3a4429fe6c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:16.349226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939fa0a-fb86-498d-9fb4-af3a4429fe6c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:16.349877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939fa0a-fb86-498d-9fb4-af3a4429fe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:16.368800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a939fa0a-fb86-498d-9fb4-af3a4429fe6c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:16.369979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a939fa0a-fb86-498d-9fb4-af3a4429fe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:16.433644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 572/1136] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:16.701372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6e2c0-b352-49c1-9fbd-cd4901a4626b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:16.710188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:16.800903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6e2c0-b352-49c1-9fbd-cd4901a4626b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:16.833887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79f7c2-ccfb-4386-9f13-37f23ca95507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:16.834506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79f7c2-ccfb-4386-9f13-37f23ca95507 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:16.834808 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79f7c2-ccfb-4386-9f13-37f23ca95507 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:16.835077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79f7c2-ccfb-4386-9f13-37f23ca95507 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:16.836171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79f7c2-ccfb-4386-9f13-37f23ca95507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:16.837584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79f7c2-ccfb-4386-9f13-37f23ca95507 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:16.839143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 565/1137] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:48:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:16.844806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939fa0a-fb86-498d-9fb4-af3a4429fe6c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:16.939706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939fa0a-fb86-498d-9fb4-af3a4429fe6c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:16.956499 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:16.972226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb48980-580e-44ea-a2e2-e75fe8212d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:16.972820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb48980-580e-44ea-a2e2-e75fe8212d71 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:16.973094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb48980-580e-44ea-a2e2-e75fe8212d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:16.973364 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb48980-580e-44ea-a2e2-e75fe8212d71 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:16.974009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb48980-580e-44ea-a2e2-e75fe8212d71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:16.975471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb48980-580e-44ea-a2e2-e75fe8212d71 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:17.077538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 573/1138] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:48:16 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:48:19.235278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec743f-c547-42bc-9f9e-91a48e80616f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:19.235841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec743f-c547-42bc-9f9e-91a48e80616f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:19.235841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec743f-c547-42bc-9f9e-91a48e80616f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:19.237128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 566/1139] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:19.242366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc373f0-885a-4e29-8604-e5f4d0b1e3af None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:19.242819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc373f0-885a-4e29-8604-e5f4d0b1e3af None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:19.242958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc373f0-885a-4e29-8604-e5f4d0b1e3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:19.260785 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-fcc373f0-885a-4e29-8604-e5f4d0b1e3af None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:19.261982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fcc373f0-885a-4e29-8604-e5f4d0b1e3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:19.327070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 574/1140] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:19.594177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec743f-c547-42bc-9f9e-91a48e80616f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:19.666157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:19.690747 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec743f-c547-42bc-9f9e-91a48e80616f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:19.723874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193b05-c424-4ff0-aeb0-97bf96123be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:19.724517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193b05-c424-4ff0-aeb0-97bf96123be8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:19.724798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193b05-c424-4ff0-aeb0-97bf96123be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:19.725071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193b05-c424-4ff0-aeb0-97bf96123be8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:19.735226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78193b05-c424-4ff0-aeb0-97bf96123be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-78193b05-c424-4ff0-aeb0-97bf96123be8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:48:19.740514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:48:19.741666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:48:19.742600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 567/1141] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:48:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:48:19.747636 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc373f0-885a-4e29-8604-e5f4d0b1e3af None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:19.842482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc373f0-885a-4e29-8604-e5f4d0b1e3af None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:19.844775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:19.874368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13be160-7233-47cf-9cdf-acd07e83c7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:19.874990 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13be160-7233-47cf-9cdf-acd07e83c7be None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:19.875269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13be160-7233-47cf-9cdf-acd07e83c7be None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:19.875542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13be160-7233-47cf-9cdf-acd07e83c7be None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:19.876192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13be160-7233-47cf-9cdf-acd07e83c7be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:19.876533 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13be160-7233-47cf-9cdf-acd07e83c7be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:19.878188 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13be160-7233-47cf-9cdf-acd07e83c7be None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:19.887078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 575/1142] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:48:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:19.892916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193b05-c424-4ff0-aeb0-97bf96123be8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:19.990919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193b05-c424-4ff0-aeb0-97bf96123be8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:20.023441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959983de-7c72-464a-b890-4c7ae38ab1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:20.024053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959983de-7c72-464a-b890-4c7ae38ab1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:20.024328 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959983de-7c72-464a-b890-4c7ae38ab1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:20.024600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959983de-7c72-464a-b890-4c7ae38ab1e6 None admin] PATH_INFO: `/v3/users/4234207fa6bf4503b4c7564222b83ae0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:20.026408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:20.037736 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959983de-7c72-464a-b890-4c7ae38ab1e6 None admin] RBAC: Authorizing `identity:get_user(user_id=4234207fa6bf4503b4c7564222b83ae0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:20.039200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959983de-7c72-464a-b890-4c7ae38ab1e6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:20.040766 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 568/1143] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:19 2022] GET /identity/v3/users/4234207fa6bf4503b4c7564222b83ae0 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:20.046556 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13be160-7233-47cf-9cdf-acd07e83c7be None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:20.143659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13be160-7233-47cf-9cdf-acd07e83c7be None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:20.175903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1c81a-dfb4-4b78-b83c-422afab5e43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:20.176498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c81a-dfb4-4b78-b83c-422afab5e43e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:20.176778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c81a-dfb4-4b78-b83c-422afab5e43e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:20.177093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c81a-dfb4-4b78-b83c-422afab5e43e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:20.209993 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:20.243439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1c81a-dfb4-4b78-b83c-422afab5e43e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:20.244859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1c81a-dfb4-4b78-b83c-422afab5e43e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:20.339034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 576/1144] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:48:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:48:20.344912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959983de-7c72-464a-b890-4c7ae38ab1e6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:20.440141 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959983de-7c72-464a-b890-4c7ae38ab1e6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:20.472241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3994b-00d0-4edb-8fcc-b97872fcdd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:20.472846 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3994b-00d0-4edb-8fcc-b97872fcdd98 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:20.473123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3994b-00d0-4edb-8fcc-b97872fcdd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:20.473391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3994b-00d0-4edb-8fcc-b97872fcdd98 None admin] PATH_INFO: `/v3/projects/b56262835e9b49b681a5dfced522c742` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:20.475034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3994b-00d0-4edb-8fcc-b97872fcdd98 None admin] RBAC: Authorizing `identity:get_project(project_id=b56262835e9b49b681a5dfced522c742)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:20.476451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3994b-00d0-4edb-8fcc-b97872fcdd98 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:20.478015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 569/1145] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:48:20 2022] GET /identity/v3/projects/b56262835e9b49b681a5dfced522c742 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:20.483830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1c81a-dfb4-4b78-b83c-422afab5e43e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:20.545724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:20.577571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1c81a-dfb4-4b78-b83c-422afab5e43e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:20.610490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e2c2e-30e3-4088-a53b-152ffed23316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:20.611100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2c2e-30e3-4088-a53b-152ffed23316 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:20.611374 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2c2e-30e3-4088-a53b-152ffed23316 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:20.611644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2c2e-30e3-4088-a53b-152ffed23316 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:20.612348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e2c2e-30e3-4088-a53b-152ffed23316 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=b56262835e9b49b681a5dfced522c742, user.id=4234207fa6bf4503b4c7564222b83ae0) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:20.612654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e2c2e-30e3-4088-a53b-152ffed23316 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:20.614056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e2c2e-30e3-4088-a53b-152ffed23316 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:20.619183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 577/1146] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:48:20 2022] GET /identity/v3/role_assignments?user.id=4234207fa6bf4503b4c7564222b83ae0&scope.project.id=b56262835e9b49b681a5dfced522c742&role.id=62eb3e525077429aacc0ada63300802e => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:22.715287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811cb56-7307-4ab9-980f-9e147796813a None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:22.715816 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811cb56-7307-4ab9-980f-9e147796813a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:22.715816 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811cb56-7307-4ab9-980f-9e147796813a None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:22.717131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 570/1147] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:22.722492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47626dbd-305c-4b55-83b5-b6d2abfe4cb4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:22.722961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47626dbd-305c-4b55-83b5-b6d2abfe4cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:22.723135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47626dbd-305c-4b55-83b5-b6d2abfe4cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:22.741607 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-47626dbd-305c-4b55-83b5-b6d2abfe4cb4 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:22.742833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-47626dbd-305c-4b55-83b5-b6d2abfe4cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:22.807898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 578/1148] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:23.072360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811cb56-7307-4ab9-980f-9e147796813a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:23.129631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:23.169095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811cb56-7307-4ab9-980f-9e147796813a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:23.201434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40248d38-c84f-4515-b7d1-5b05e722df8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:23.202044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40248d38-c84f-4515-b7d1-5b05e722df8c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:23.202319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40248d38-c84f-4515-b7d1-5b05e722df8c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:23.202588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40248d38-c84f-4515-b7d1-5b05e722df8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:23.212830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40248d38-c84f-4515-b7d1-5b05e722df8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-40248d38-c84f-4515-b7d1-5b05e722df8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:48:23.217535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:48:23.218626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:48:23.219894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 571/1149] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:48:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:48:23.224527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47626dbd-305c-4b55-83b5-b6d2abfe4cb4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:23.281333 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:23.321206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47626dbd-305c-4b55-83b5-b6d2abfe4cb4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:23.353486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5c772-654a-4772-b7df-e7f69105583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:23.354025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5c772-654a-4772-b7df-e7f69105583d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:23.354306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5c772-654a-4772-b7df-e7f69105583d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:23.354580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5c772-654a-4772-b7df-e7f69105583d None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:23.355267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5c772-654a-4772-b7df-e7f69105583d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:23.355573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5c772-654a-4772-b7df-e7f69105583d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:23.356921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5c772-654a-4772-b7df-e7f69105583d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:23.364595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 579/1150] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:48:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:23.370730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40248d38-c84f-4515-b7d1-5b05e722df8c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:23.465918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40248d38-c84f-4515-b7d1-5b05e722df8c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:23.485348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:23.498241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315ea2b-ce65-44c7-a50a-2a97c1098033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:23.498914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ea2b-ce65-44c7-a50a-2a97c1098033 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:23.499146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ea2b-ce65-44c7-a50a-2a97c1098033 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:23.499414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ea2b-ce65-44c7-a50a-2a97c1098033 None admin] PATH_INFO: `/v3/users/4234207fa6bf4503b4c7564222b83ae0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:23.500519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315ea2b-ce65-44c7-a50a-2a97c1098033 None admin] RBAC: Authorizing `identity:get_user(user_id=4234207fa6bf4503b4c7564222b83ae0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:23.501938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315ea2b-ce65-44c7-a50a-2a97c1098033 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:23.503499 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 572/1151] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:23 2022] GET /identity/v3/users/4234207fa6bf4503b4c7564222b83ae0 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:23.509213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5c772-654a-4772-b7df-e7f69105583d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:23.606191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5c772-654a-4772-b7df-e7f69105583d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:23.627251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:23.638208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19d375-3461-4053-ac76-dce938fdd618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:23.638854 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19d375-3461-4053-ac76-dce938fdd618 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:23.639136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19d375-3461-4053-ac76-dce938fdd618 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:23.639410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19d375-3461-4053-ac76-dce938fdd618 None admin] PATH_INFO: `/v3/projects/b56262835e9b49b681a5dfced522c742` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:23.640511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19d375-3461-4053-ac76-dce938fdd618 None admin] RBAC: Authorizing `identity:get_project(project_id=b56262835e9b49b681a5dfced522c742)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:23.641930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19d375-3461-4053-ac76-dce938fdd618 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:23.643536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 580/1152] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:48:23 2022] GET /identity/v3/projects/b56262835e9b49b681a5dfced522c742 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:23.649673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315ea2b-ce65-44c7-a50a-2a97c1098033 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:23.745096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315ea2b-ce65-44c7-a50a-2a97c1098033 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:23.777681 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb136c18-ad08-4e0f-baa1-733ff38f450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:23.778323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb136c18-ad08-4e0f-baa1-733ff38f450b None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:23.778599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb136c18-ad08-4e0f-baa1-733ff38f450b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:23.778892 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb136c18-ad08-4e0f-baa1-733ff38f450b None admin] PATH_INFO: `/v3/projects/b56262835e9b49b681a5dfced522c742/users/4234207fa6bf4503b4c7564222b83ae0/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:23.780753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb136c18-ad08-4e0f-baa1-733ff38f450b None admin] RBAC: Authorizing `identity:create_grant(project_id=b56262835e9b49b681a5dfced522c742, user_id=4234207fa6bf4503b4c7564222b83ae0, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:23.782177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb136c18-ad08-4e0f-baa1-733ff38f450b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:23.871257 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 573/1153] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:48:23 2022] PUT /identity/v3/projects/b56262835e9b49b681a5dfced522c742/users/4234207fa6bf4503b4c7564222b83ae0/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:48:25.965797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c189e93-3bec-435f-9ece-60247b7fedc1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:25.966339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c189e93-3bec-435f-9ece-60247b7fedc1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:25.966339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c189e93-3bec-435f-9ece-60247b7fedc1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:25.967720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 581/1154] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:25.972814 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68152a7b-771e-4ae1-af3c-38a7532256b7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:25.973226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68152a7b-771e-4ae1-af3c-38a7532256b7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:25.973365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68152a7b-771e-4ae1-af3c-38a7532256b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:25.991586 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-68152a7b-771e-4ae1-af3c-38a7532256b7 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:25.992707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-68152a7b-771e-4ae1-af3c-38a7532256b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:26.034272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:26.058173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 574/1155] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:26.325946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c189e93-3bec-435f-9ece-60247b7fedc1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:26.425179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c189e93-3bec-435f-9ece-60247b7fedc1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:26.456891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12212-5a03-4be9-8dd9-553c4830824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:26.457506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12212-5a03-4be9-8dd9-553c4830824a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:26.457781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12212-5a03-4be9-8dd9-553c4830824a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:26.458049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12212-5a03-4be9-8dd9-553c4830824a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:26.468685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12212-5a03-4be9-8dd9-553c4830824a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-00c12212-5a03-4be9-8dd9-553c4830824a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:48:26.473389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:48:26.478733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:48:26.479955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 582/1156] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:48:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:48:26.480873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68152a7b-771e-4ae1-af3c-38a7532256b7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:26.574644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68152a7b-771e-4ae1-af3c-38a7532256b7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:26.607600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511005d-46e8-4566-9185-4aff0948481a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:26.608132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511005d-46e8-4566-9185-4aff0948481a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:26.608405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511005d-46e8-4566-9185-4aff0948481a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:26.608674 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511005d-46e8-4566-9185-4aff0948481a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:26.609324 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511005d-46e8-4566-9185-4aff0948481a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:26.609624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511005d-46e8-4566-9185-4aff0948481a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:26.610999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511005d-46e8-4566-9185-4aff0948481a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:26.618623 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 575/1157] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:26.624251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12212-5a03-4be9-8dd9-553c4830824a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:26.672732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:26.720153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12212-5a03-4be9-8dd9-553c4830824a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:26.752835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6330b-7064-4515-93fe-91d83d500479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:26.753451 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6330b-7064-4515-93fe-91d83d500479 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:26.753725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6330b-7064-4515-93fe-91d83d500479 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:26.753999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6330b-7064-4515-93fe-91d83d500479 None admin] PATH_INFO: `/v3/users/4234207fa6bf4503b4c7564222b83ae0` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:26.755148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6330b-7064-4515-93fe-91d83d500479 None admin] RBAC: Authorizing `identity:get_user(user_id=4234207fa6bf4503b4c7564222b83ae0)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:26.756587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6330b-7064-4515-93fe-91d83d500479 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:26.758164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 583/1158] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:26 2022] GET /identity/v3/users/4234207fa6bf4503b4c7564222b83ae0 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:26.763836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511005d-46e8-4566-9185-4aff0948481a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:26.804828 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:26.860981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511005d-46e8-4566-9185-4aff0948481a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:26.893137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade4788-8f7d-453e-9412-ba972376eae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:26.896415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade4788-8f7d-453e-9412-ba972376eae5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:26.896415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade4788-8f7d-453e-9412-ba972376eae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:26.896415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade4788-8f7d-453e-9412-ba972376eae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:26.958961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade4788-8f7d-453e-9412-ba972376eae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:26.960395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade4788-8f7d-453e-9412-ba972376eae5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:26.998251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:27.054154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 576/1159] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:48:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:48:27.060281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6330b-7064-4515-93fe-91d83d500479 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:27.157204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6330b-7064-4515-93fe-91d83d500479 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:27.173147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:27.189632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e49d8-6c2d-431a-8a1a-07952ecba5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:27.190257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e49d8-6c2d-431a-8a1a-07952ecba5eb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:27.190536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e49d8-6c2d-431a-8a1a-07952ecba5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:27.190845 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e49d8-6c2d-431a-8a1a-07952ecba5eb None admin] PATH_INFO: `/v3/projects/b56262835e9b49b681a5dfced522c742` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:27.191957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e49d8-6c2d-431a-8a1a-07952ecba5eb None admin] RBAC: Authorizing `identity:get_project(project_id=b56262835e9b49b681a5dfced522c742)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:27.193924 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e49d8-6c2d-431a-8a1a-07952ecba5eb None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:27.195547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 584/1160] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:48:27 2022] GET /identity/v3/projects/b56262835e9b49b681a5dfced522c742 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:27.201631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade4788-8f7d-453e-9412-ba972376eae5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:27.295502 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade4788-8f7d-453e-9412-ba972376eae5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:27.329321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff991a-6dc3-4f68-826f-1410ed303f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:27.329915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff991a-6dc3-4f68-826f-1410ed303f2b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:27.330197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff991a-6dc3-4f68-826f-1410ed303f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:27.330471 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff991a-6dc3-4f68-826f-1410ed303f2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:27.331226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff991a-6dc3-4f68-826f-1410ed303f2b None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=b56262835e9b49b681a5dfced522c742, user.id=4234207fa6bf4503b4c7564222b83ae0) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:27.331537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff991a-6dc3-4f68-826f-1410ed303f2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:27.332958 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff991a-6dc3-4f68-826f-1410ed303f2b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:27.338577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 577/1161] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:48:27 2022] GET /identity/v3/role_assignments?user.id=4234207fa6bf4503b4c7564222b83ae0&scope.project.id=b56262835e9b49b681a5dfced522c742&role.id=62eb3e525077429aacc0ada63300802e => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:29.476907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85bc00-2539-431c-9ac2-c738907086e9 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:29.477436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85bc00-2539-431c-9ac2-c738907086e9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:29.477436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85bc00-2539-431c-9ac2-c738907086e9 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:29.478824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 585/1162] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:29.483993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68057bb7-b5d0-42e8-9094-720b43928dd1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:29.484437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68057bb7-b5d0-42e8-9094-720b43928dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:29.484532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68057bb7-b5d0-42e8-9094-720b43928dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:29.491772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:29.502253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-68057bb7-b5d0-42e8-9094-720b43928dd1 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:29.503468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-68057bb7-b5d0-42e8-9094-720b43928dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:29.567592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 578/1163] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:29.834044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85bc00-2539-431c-9ac2-c738907086e9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:29.933422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85bc00-2539-431c-9ac2-c738907086e9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:29.966785 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ed05c-428a-4e5f-a188-44d85463ea6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:29.967384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed05c-428a-4e5f-a188-44d85463ea6d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:29.967664 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed05c-428a-4e5f-a188-44d85463ea6d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:29.967939 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed05c-428a-4e5f-a188-44d85463ea6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:30.020485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:30.033647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ed05c-428a-4e5f-a188-44d85463ea6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:30.035195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ed05c-428a-4e5f-a188-44d85463ea6d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:30.130654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 586/1164] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:48:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:48:30.136648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68057bb7-b5d0-42e8-9094-720b43928dd1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:30.231940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68057bb7-b5d0-42e8-9094-720b43928dd1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:30.264989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643a910-7c9e-4fde-b6c5-dc2cacc9cc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:30.265604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643a910-7c9e-4fde-b6c5-dc2cacc9cc4d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:30.265882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643a910-7c9e-4fde-b6c5-dc2cacc9cc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:30.266153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643a910-7c9e-4fde-b6c5-dc2cacc9cc4d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:30.267975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:30.273227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643a910-7c9e-4fde-b6c5-dc2cacc9cc4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:30.274602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643a910-7c9e-4fde-b6c5-dc2cacc9cc4d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a643a910-7c9e-4fde-b6c5-dc2cacc9cc4d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:48:30.285812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:48:30.291864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:48:30.293012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 579/1165] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Sep 16 07:48:30 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:48:30.293889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ed05c-428a-4e5f-a188-44d85463ea6d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:30.375359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:30.391443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ed05c-428a-4e5f-a188-44d85463ea6d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:30.424403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9e165-bda0-41db-a4a3-24190890ad87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:30.425009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9e165-bda0-41db-a4a3-24190890ad87 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:30.425295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9e165-bda0-41db-a4a3-24190890ad87 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:30.425573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9e165-bda0-41db-a4a3-24190890ad87 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:30.426214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9e165-bda0-41db-a4a3-24190890ad87 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:30.426520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9e165-bda0-41db-a4a3-24190890ad87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:30.428426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9e165-bda0-41db-a4a3-24190890ad87 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:30.435610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 587/1166] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Sep 16 07:48:30 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:32.551187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27aec12-dc85-4611-a052-a6a17568ca57 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:32.552257 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27aec12-dc85-4611-a052-a6a17568ca57 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:32.552257 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27aec12-dc85-4611-a052-a6a17568ca57 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:32.553562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 580/1167] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:32.558673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a727-4ed2-49d9-9249-1e1fa8a08bb0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:32.559080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a727-4ed2-49d9-9249-1e1fa8a08bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:32.559266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a727-4ed2-49d9-9249-1e1fa8a08bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:32.577875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6099a727-4ed2-49d9-9249-1e1fa8a08bb0 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:32.579104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6099a727-4ed2-49d9-9249-1e1fa8a08bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:32.644122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 588/1168] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:32.907166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27aec12-dc85-4611-a052-a6a17568ca57 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:33.004901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27aec12-dc85-4611-a052-a6a17568ca57 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:33.036734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f1991-14f5-4e39-bb28-c2cbfc073820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:33.037321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f1991-14f5-4e39-bb28-c2cbfc073820 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:33.037596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f1991-14f5-4e39-bb28-c2cbfc073820 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:33.037861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f1991-14f5-4e39-bb28-c2cbfc073820 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:33.038507 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f1991-14f5-4e39-bb28-c2cbfc073820 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:33.039890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f1991-14f5-4e39-bb28-c2cbfc073820 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:33.123990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 581/1169] 10.222.0.59 () {60 vars in 1138 bytes} [Fri Sep 16 07:48:32 2022] POST /identity/v3/domains => generated 267 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:48:35.268135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa91bb-2ed2-4de8-95b9-955624ec0582 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:35.268713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa91bb-2ed2-4de8-95b9-955624ec0582 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:35.268713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa91bb-2ed2-4de8-95b9-955624ec0582 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:35.270024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 589/1170] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:35.275893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac52431-f273-4daa-9425-682a3428424f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:35.276287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac52431-f273-4daa-9425-682a3428424f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:35.276445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac52431-f273-4daa-9425-682a3428424f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:35.294391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0ac52431-f273-4daa-9425-682a3428424f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:35.295626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0ac52431-f273-4daa-9425-682a3428424f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:35.343738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:35.360239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 582/1171] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:35.624731 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa91bb-2ed2-4de8-95b9-955624ec0582 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:35.702768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:35.721378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa91bb-2ed2-4de8-95b9-955624ec0582 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:35.755361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0252d-d0c1-483f-9d60-2a1f89be0082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:35.755982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0252d-d0c1-483f-9d60-2a1f89be0082 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:35.756256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0252d-d0c1-483f-9d60-2a1f89be0082 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:35.756524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0252d-d0c1-483f-9d60-2a1f89be0082 None admin] PATH_INFO: `/v3/domains/a8946fae4af740f08770a61dfbd3d4da` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:35.766439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0252d-d0c1-483f-9d60-2a1f89be0082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a8946fae4af740f08770a61dfbd3d4da)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:35.767896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0252d-d0c1-483f-9d60-2a1f89be0082 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:35.769474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 590/1172] 10.222.0.59 () {56 vars in 1183 bytes} [Fri Sep 16 07:48:35 2022] GET /identity/v3/domains/a8946fae4af740f08770a61dfbd3d4da => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:35.775228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac52431-f273-4daa-9425-682a3428424f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:35.871189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac52431-f273-4daa-9425-682a3428424f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:35.904304 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a703d1-e900-4f3c-91d6-2a035ad457e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:35.904847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a703d1-e900-4f3c-91d6-2a035ad457e6 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:35.905126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a703d1-e900-4f3c-91d6-2a035ad457e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:35.905396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a703d1-e900-4f3c-91d6-2a035ad457e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:35.906054 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a703d1-e900-4f3c-91d6-2a035ad457e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:35.907538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a703d1-e900-4f3c-91d6-2a035ad457e6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:36.025448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 583/1173] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:48:35 2022] POST /identity/v3/projects => generated 361 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:48:38.158741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab37743-4592-403a-afaf-175d0d5170e5 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:38.159236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab37743-4592-403a-afaf-175d0d5170e5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:38.159236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab37743-4592-403a-afaf-175d0d5170e5 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:38.160546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 591/1174] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:38.166006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b94e-60b2-4027-b9ae-d85b0c7e956c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:38.166433 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b94e-60b2-4027-b9ae-d85b0c7e956c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:38.166550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b94e-60b2-4027-b9ae-d85b0c7e956c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:38.184313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3444b94e-60b2-4027-b9ae-d85b0c7e956c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:38.185761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3444b94e-60b2-4027-b9ae-d85b0c7e956c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:38.214311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:38.250532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 584/1175] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:38.520536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab37743-4592-403a-afaf-175d0d5170e5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:38.619262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab37743-4592-403a-afaf-175d0d5170e5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:38.650822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9175b8-dfc5-49c4-82d5-2436194a23c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:38.651438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9175b8-dfc5-49c4-82d5-2436194a23c6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:38.651714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9175b8-dfc5-49c4-82d5-2436194a23c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:38.651981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9175b8-dfc5-49c4-82d5-2436194a23c6 None admin] PATH_INFO: `/v3/domains/a8946fae4af740f08770a61dfbd3d4da` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:38.653055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9175b8-dfc5-49c4-82d5-2436194a23c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a8946fae4af740f08770a61dfbd3d4da)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:38.654470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9175b8-dfc5-49c4-82d5-2436194a23c6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:38.656046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 592/1176] 10.222.0.59 () {56 vars in 1183 bytes} [Fri Sep 16 07:48:38 2022] GET /identity/v3/domains/a8946fae4af740f08770a61dfbd3d4da => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:38.661623 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444b94e-60b2-4027-b9ae-d85b0c7e956c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:38.756954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444b94e-60b2-4027-b9ae-d85b0c7e956c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:38.789645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2555c-7a58-446f-9623-ce4f3ede52a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:38.790182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2555c-7a58-446f-9623-ce4f3ede52a6 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:38.790458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2555c-7a58-446f-9623-ce4f3ede52a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:38.790747 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2555c-7a58-446f-9623-ce4f3ede52a6 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:38.791397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2555c-7a58-446f-9623-ce4f3ede52a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:38.792814 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2555c-7a58-446f-9623-ce4f3ede52a6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:38.898515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 585/1177] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:48:38 2022] POST /identity/v3/users => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:48:41.088411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48efad8-c3b1-4bc5-b57d-16eebc278520 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:41.088934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48efad8-c3b1-4bc5-b57d-16eebc278520 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:41.088934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48efad8-c3b1-4bc5-b57d-16eebc278520 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:41.090214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 593/1178] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:41.095485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bbeab-7a61-437f-8332-850a1211fadf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:41.095911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bbeab-7a61-437f-8332-850a1211fadf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:41.096045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bbeab-7a61-437f-8332-850a1211fadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:41.113913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-995bbeab-7a61-437f-8332-850a1211fadf None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:41.115142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-995bbeab-7a61-437f-8332-850a1211fadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:41.131650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:41.179918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 586/1179] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:41.448281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48efad8-c3b1-4bc5-b57d-16eebc278520 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:41.459493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:41.546919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48efad8-c3b1-4bc5-b57d-16eebc278520 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:41.579998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58618d-bf9e-4b88-afac-a349e4cf7bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:41.580620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58618d-bf9e-4b88-afac-a349e4cf7bfc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:41.580902 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58618d-bf9e-4b88-afac-a349e4cf7bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:41.581174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58618d-bf9e-4b88-afac-a349e4cf7bfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:41.591374 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58618d-bf9e-4b88-afac-a349e4cf7bfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-1c58618d-bf9e-4b88-afac-a349e4cf7bfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:48:41.596133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:48:41.597332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:48:41.598429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 594/1180] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:48:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:48:41.603138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bbeab-7a61-437f-8332-850a1211fadf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:41.698942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bbeab-7a61-437f-8332-850a1211fadf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:41.732081 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa7456-ac7e-4037-9a2e-22beedcd11b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:41.732677 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa7456-ac7e-4037-9a2e-22beedcd11b7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:41.732957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa7456-ac7e-4037-9a2e-22beedcd11b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:41.733228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa7456-ac7e-4037-9a2e-22beedcd11b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:41.733884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa7456-ac7e-4037-9a2e-22beedcd11b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:41.734184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa7456-ac7e-4037-9a2e-22beedcd11b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:41.735589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa7456-ac7e-4037-9a2e-22beedcd11b7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:41.745842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 587/1181] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:48:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:41.751727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58618d-bf9e-4b88-afac-a349e4cf7bfc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:41.817426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:41.850148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58618d-bf9e-4b88-afac-a349e4cf7bfc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:41.883079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bc805-2454-4590-a38b-b9ec7f959ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:41.883741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bc805-2454-4590-a38b-b9ec7f959ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:41.884017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bc805-2454-4590-a38b-b9ec7f959ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:41.891022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bc805-2454-4590-a38b-b9ec7f959ab8 None admin] PATH_INFO: `/v3/users/aee4f10792b54a6996c8c03fcf8f355c` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:41.897189 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bc805-2454-4590-a38b-b9ec7f959ab8 None admin] RBAC: Authorizing `identity:get_user(user_id=aee4f10792b54a6996c8c03fcf8f355c)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:41.898701 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bc805-2454-4590-a38b-b9ec7f959ab8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:41.900288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 595/1182] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:41 2022] GET /identity/v3/users/aee4f10792b54a6996c8c03fcf8f355c => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:41.905985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa7456-ac7e-4037-9a2e-22beedcd11b7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:41.922098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:42.002267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa7456-ac7e-4037-9a2e-22beedcd11b7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:42.033429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8aec09-3d1a-4d0f-b9d0-91864427bc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:42.034016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8aec09-3d1a-4d0f-b9d0-91864427bc78 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:42.034294 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8aec09-3d1a-4d0f-b9d0-91864427bc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:42.034563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8aec09-3d1a-4d0f-b9d0-91864427bc78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:42.098957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8aec09-3d1a-4d0f-b9d0-91864427bc78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:42.100386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8aec09-3d1a-4d0f-b9d0-91864427bc78 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:42.113766 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:42.193320 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 588/1183] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:48:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:48:42.199507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bc805-2454-4590-a38b-b9ec7f959ab8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:42.294673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bc805-2454-4590-a38b-b9ec7f959ab8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:42.319851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:42.328023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53551a15-06fa-4fb9-9cd8-3bfc4d865a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:42.328635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53551a15-06fa-4fb9-9cd8-3bfc4d865a97 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:42.328908 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53551a15-06fa-4fb9-9cd8-3bfc4d865a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:42.329173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53551a15-06fa-4fb9-9cd8-3bfc4d865a97 None admin] PATH_INFO: `/v3/projects/b3b16abf185544d882d0de0b839a8ec7` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:42.330259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53551a15-06fa-4fb9-9cd8-3bfc4d865a97 None admin] RBAC: Authorizing `identity:get_project(project_id=b3b16abf185544d882d0de0b839a8ec7)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:42.331686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53551a15-06fa-4fb9-9cd8-3bfc4d865a97 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:42.333824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 596/1184] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:48:42 2022] GET /identity/v3/projects/b3b16abf185544d882d0de0b839a8ec7 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:42.339840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8aec09-3d1a-4d0f-b9d0-91864427bc78 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:42.434531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8aec09-3d1a-4d0f-b9d0-91864427bc78 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:42.447947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:42.467424 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f7211-e0f7-4fcd-915d-55ed33a411c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:42.468014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f7211-e0f7-4fcd-915d-55ed33a411c7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:42.468291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f7211-e0f7-4fcd-915d-55ed33a411c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:42.468560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f7211-e0f7-4fcd-915d-55ed33a411c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:42.469279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f7211-e0f7-4fcd-915d-55ed33a411c7 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=b3b16abf185544d882d0de0b839a8ec7, user.id=aee4f10792b54a6996c8c03fcf8f355c) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:42.469582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f7211-e0f7-4fcd-915d-55ed33a411c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:42.471032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f7211-e0f7-4fcd-915d-55ed33a411c7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:42.477406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 589/1185] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:48:42 2022] GET /identity/v3/role_assignments?user.id=aee4f10792b54a6996c8c03fcf8f355c&scope.project.id=b3b16abf185544d882d0de0b839a8ec7&role.id=62eb3e525077429aacc0ada63300802e => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:44.560293 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fd56-a129-426a-af16-e47f0807b6ee None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:44.560848 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fd56-a129-426a-af16-e47f0807b6ee None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:44.560848 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fd56-a129-426a-af16-e47f0807b6ee None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:44.562155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 597/1186] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:44.567240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880301d0-09a1-4dc6-8338-31f541659348 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:44.567622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880301d0-09a1-4dc6-8338-31f541659348 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:44.567780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880301d0-09a1-4dc6-8338-31f541659348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:44.585958 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-880301d0-09a1-4dc6-8338-31f541659348 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:44.587167 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-880301d0-09a1-4dc6-8338-31f541659348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:44.651142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 590/1187] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:44.919437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63fd56-a129-426a-af16-e47f0807b6ee None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:45.018203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63fd56-a129-426a-af16-e47f0807b6ee None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:45.050669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf5320-f9e9-413c-bf2e-dccdf9cff510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:45.051347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf5320-f9e9-413c-bf2e-dccdf9cff510 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:45.051625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf5320-f9e9-413c-bf2e-dccdf9cff510 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:45.051893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf5320-f9e9-413c-bf2e-dccdf9cff510 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:45.061799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf5320-f9e9-413c-bf2e-dccdf9cff510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d6cf5320-f9e9-413c-bf2e-dccdf9cff510 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:48:45.066525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:48:45.068096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:48:45.069022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 598/1188] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:48:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:48:45.074065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880301d0-09a1-4dc6-8338-31f541659348 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:45.167740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880301d0-09a1-4dc6-8338-31f541659348 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:45.177150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:45.200973 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95069a5-4abd-46c7-9689-ae50d93fe532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:45.201588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95069a5-4abd-46c7-9689-ae50d93fe532 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:45.201866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95069a5-4abd-46c7-9689-ae50d93fe532 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:45.202138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95069a5-4abd-46c7-9689-ae50d93fe532 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:45.202894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95069a5-4abd-46c7-9689-ae50d93fe532 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:45.203201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95069a5-4abd-46c7-9689-ae50d93fe532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:45.204560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95069a5-4abd-46c7-9689-ae50d93fe532 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:45.212578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 591/1189] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:48:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:45.218187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf5320-f9e9-413c-bf2e-dccdf9cff510 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:45.272685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:45.315224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf5320-f9e9-413c-bf2e-dccdf9cff510 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:45.347523 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19570fd-c7b7-43e0-8cb4-6e2cc410b100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:45.348134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19570fd-c7b7-43e0-8cb4-6e2cc410b100 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:45.348415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19570fd-c7b7-43e0-8cb4-6e2cc410b100 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:45.348689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19570fd-c7b7-43e0-8cb4-6e2cc410b100 None admin] PATH_INFO: `/v3/users/aee4f10792b54a6996c8c03fcf8f355c` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:45.349794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19570fd-c7b7-43e0-8cb4-6e2cc410b100 None admin] RBAC: Authorizing `identity:get_user(user_id=aee4f10792b54a6996c8c03fcf8f355c)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:45.351275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19570fd-c7b7-43e0-8cb4-6e2cc410b100 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:45.352819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 599/1190] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:45 2022] GET /identity/v3/users/aee4f10792b54a6996c8c03fcf8f355c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:45.358370 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95069a5-4abd-46c7-9689-ae50d93fe532 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:45.455017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95069a5-4abd-46c7-9689-ae50d93fe532 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:45.486600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b92a4e-549d-4214-af4f-dc51fee55e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:45.487239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b92a4e-549d-4214-af4f-dc51fee55e89 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:45.487519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b92a4e-549d-4214-af4f-dc51fee55e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:45.487794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b92a4e-549d-4214-af4f-dc51fee55e89 None admin] PATH_INFO: `/v3/projects/b3b16abf185544d882d0de0b839a8ec7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:45.489429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b92a4e-549d-4214-af4f-dc51fee55e89 None admin] RBAC: Authorizing `identity:get_project(project_id=b3b16abf185544d882d0de0b839a8ec7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:45.490879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b92a4e-549d-4214-af4f-dc51fee55e89 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:45.492457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 592/1191] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:48:45 2022] GET /identity/v3/projects/b3b16abf185544d882d0de0b839a8ec7 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:45.498067 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19570fd-c7b7-43e0-8cb4-6e2cc410b100 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:45.593496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19570fd-c7b7-43e0-8cb4-6e2cc410b100 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:45.615880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:45.626729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ebe74-13f2-4b54-a66d-f9082370d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:45.629208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ebe74-13f2-4b54-a66d-f9082370d8f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:45.629208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ebe74-13f2-4b54-a66d-f9082370d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:45.629208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ebe74-13f2-4b54-a66d-f9082370d8f7 None admin] PATH_INFO: `/v3/projects/b3b16abf185544d882d0de0b839a8ec7/users/aee4f10792b54a6996c8c03fcf8f355c/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:45.629916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ebe74-13f2-4b54-a66d-f9082370d8f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3b16abf185544d882d0de0b839a8ec7, user_id=aee4f10792b54a6996c8c03fcf8f355c, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:45.631360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ebe74-13f2-4b54-a66d-f9082370d8f7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:45.699133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 600/1192] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:48:45 2022] PUT /identity/v3/projects/b3b16abf185544d882d0de0b839a8ec7/users/aee4f10792b54a6996c8c03fcf8f355c/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:48:47.785277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cb8e1-78a3-451d-bfdc-473ea222cfea None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:47.785796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cb8e1-78a3-451d-bfdc-473ea222cfea None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:47.785887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cb8e1-78a3-451d-bfdc-473ea222cfea None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:47.787836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 593/1193] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:48:47.792945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01648a65-103a-4898-b1b5-fd0a6182a070 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:47.793368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01648a65-103a-4898-b1b5-fd0a6182a070 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:47.793492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01648a65-103a-4898-b1b5-fd0a6182a070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:47.811375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-01648a65-103a-4898-b1b5-fd0a6182a070 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:48:47.812553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-01648a65-103a-4898-b1b5-fd0a6182a070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:48:47.876884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 601/1194] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:48:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:48:48.144756 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cb8e1-78a3-451d-bfdc-473ea222cfea None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:48.185417 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:48.245301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cb8e1-78a3-451d-bfdc-473ea222cfea None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:48.277864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f00bf8-f3ad-411d-a1c8-5bbfc571c772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:48.278482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00bf8-f3ad-411d-a1c8-5bbfc571c772 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:48.278789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00bf8-f3ad-411d-a1c8-5bbfc571c772 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:48.279066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00bf8-f3ad-411d-a1c8-5bbfc571c772 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:48.289384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f00bf8-f3ad-411d-a1c8-5bbfc571c772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-95f00bf8-f3ad-411d-a1c8-5bbfc571c772 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:48:48.294140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:48:48.295260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:48:48.296184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 594/1195] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:48:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:48:48.301667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01648a65-103a-4898-b1b5-fd0a6182a070 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:48.395011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01648a65-103a-4898-b1b5-fd0a6182a070 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:48.407230 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:48.427585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc402ca4-be6c-435d-9a83-6b59b019adb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:48.428210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc402ca4-be6c-435d-9a83-6b59b019adb0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:48.428486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc402ca4-be6c-435d-9a83-6b59b019adb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:48.428753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc402ca4-be6c-435d-9a83-6b59b019adb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:48.429398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc402ca4-be6c-435d-9a83-6b59b019adb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:48.429696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc402ca4-be6c-435d-9a83-6b59b019adb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:48.431074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc402ca4-be6c-435d-9a83-6b59b019adb0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:48.439430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 602/1196] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:48:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:48.446083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f00bf8-f3ad-411d-a1c8-5bbfc571c772 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:48.539051 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:48.542712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f00bf8-f3ad-411d-a1c8-5bbfc571c772 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:48.575024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40f67d-5b7d-476e-8c32-0ce44216c6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:48.575637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40f67d-5b7d-476e-8c32-0ce44216c6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:48.575913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40f67d-5b7d-476e-8c32-0ce44216c6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:48.576181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40f67d-5b7d-476e-8c32-0ce44216c6b3 None admin] PATH_INFO: `/v3/users/aee4f10792b54a6996c8c03fcf8f355c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:48.577271 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40f67d-5b7d-476e-8c32-0ce44216c6b3 None admin] RBAC: Authorizing `identity:get_user(user_id=aee4f10792b54a6996c8c03fcf8f355c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:48.578699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40f67d-5b7d-476e-8c32-0ce44216c6b3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:48.580251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 595/1197] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Sep 16 07:48:48 2022] GET /identity/v3/users/aee4f10792b54a6996c8c03fcf8f355c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:48.585988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc402ca4-be6c-435d-9a83-6b59b019adb0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:48.682941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc402ca4-be6c-435d-9a83-6b59b019adb0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:48.714487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503fa77-2298-46f5-8894-d966a602dfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:48.715127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503fa77-2298-46f5-8894-d966a602dfef None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:48.715406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503fa77-2298-46f5-8894-d966a602dfef None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:48.715678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503fa77-2298-46f5-8894-d966a602dfef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:48.757077 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:48.780893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503fa77-2298-46f5-8894-d966a602dfef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:48.782368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503fa77-2298-46f5-8894-d966a602dfef None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:48.877781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 603/1198] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:48:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:48:48.883752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40f67d-5b7d-476e-8c32-0ce44216c6b3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:48.979075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40f67d-5b7d-476e-8c32-0ce44216c6b3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:49.010215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3ed74-819e-43e9-9125-938d3de4deca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:49.010863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3ed74-819e-43e9-9125-938d3de4deca None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:49.011140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3ed74-819e-43e9-9125-938d3de4deca None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:49.011410 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3ed74-819e-43e9-9125-938d3de4deca None admin] PATH_INFO: `/v3/projects/b3b16abf185544d882d0de0b839a8ec7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:49.012506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3ed74-819e-43e9-9125-938d3de4deca None admin] RBAC: Authorizing `identity:get_project(project_id=b3b16abf185544d882d0de0b839a8ec7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:49.013918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3ed74-819e-43e9-9125-938d3de4deca None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:49.015509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 596/1199] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Sep 16 07:48:48 2022] GET /identity/v3/projects/b3b16abf185544d882d0de0b839a8ec7 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:49.021909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503fa77-2298-46f5-8894-d966a602dfef None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:48:49.093736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:49.117758 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503fa77-2298-46f5-8894-d966a602dfef None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:48:49.151206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d1a2e-b998-4212-a3ba-f9f0bd71583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:48:49.151798 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1a2e-b998-4212-a3ba-f9f0bd71583f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:48:49.152078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1a2e-b998-4212-a3ba-f9f0bd71583f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:48:49.152352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1a2e-b998-4212-a3ba-f9f0bd71583f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:48:49.153073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d1a2e-b998-4212-a3ba-f9f0bd71583f None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=b3b16abf185544d882d0de0b839a8ec7, user.id=aee4f10792b54a6996c8c03fcf8f355c) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:48:49.153381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d1a2e-b998-4212-a3ba-f9f0bd71583f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:48:49.154821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d1a2e-b998-4212-a3ba-f9f0bd71583f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:48:49.160925 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 604/1200] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:48:49 2022] GET /identity/v3/role_assignments?user.id=aee4f10792b54a6996c8c03fcf8f355c&scope.project.id=b3b16abf185544d882d0de0b839a8ec7&role.id=62eb3e525077429aacc0ada63300802e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:48:59.006309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=83 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:48:59.157859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=31 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:01.092534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81134f3f-9c12-4b8b-a764-1c1303b6e310 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:01.093088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81134f3f-9c12-4b8b-a764-1c1303b6e310 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:01.093088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81134f3f-9c12-4b8b-a764-1c1303b6e310 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:01.094374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 597/1201] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:50:01.099678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2013c-80f2-4fa2-8f5d-5caae4b495c0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:01.100129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2013c-80f2-4fa2-8f5d-5caae4b495c0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:01.100235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2013c-80f2-4fa2-8f5d-5caae4b495c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:01.120275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-01a2013c-80f2-4fa2-8f5d-5caae4b495c0 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:50:01.121470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-01a2013c-80f2-4fa2-8f5d-5caae4b495c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:50:01.186798 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 605/1202] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:50:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:50:01.457488 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81134f3f-9c12-4b8b-a764-1c1303b6e310 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:01.558652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81134f3f-9c12-4b8b-a764-1c1303b6e310 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:01.592117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0929bd5-a3af-4952-8078-862276c3a20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:01.592741 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0929bd5-a3af-4952-8078-862276c3a20e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:01.593019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0929bd5-a3af-4952-8078-862276c3a20e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:01.593290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0929bd5-a3af-4952-8078-862276c3a20e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:01.604206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0929bd5-a3af-4952-8078-862276c3a20e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:01.605642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0929bd5-a3af-4952-8078-862276c3a20e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c0929bd5-a3af-4952-8078-862276c3a20e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:01.620310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:01.621401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:01.622561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:01.622561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:01.622561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:01.622561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 598/1203] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:01.627879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2013c-80f2-4fa2-8f5d-5caae4b495c0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:01.726661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2013c-80f2-4fa2-8f5d-5caae4b495c0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:01.754834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:01.760953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d3ccc-cffe-4b66-9630-8b9d962cf198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:01.761562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d3ccc-cffe-4b66-9630-8b9d962cf198 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:01.761846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d3ccc-cffe-4b66-9630-8b9d962cf198 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:01.762122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d3ccc-cffe-4b66-9630-8b9d962cf198 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:01.762817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d3ccc-cffe-4b66-9630-8b9d962cf198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:01.763133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d3ccc-cffe-4b66-9630-8b9d962cf198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:01.764492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d3ccc-cffe-4b66-9630-8b9d962cf198 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:01.776095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 606/1204] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:01.781876 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0929bd5-a3af-4952-8078-862276c3a20e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:01.833563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:01.879843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0929bd5-a3af-4952-8078-862276c3a20e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:01.912159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee48ae0-a50d-4b30-83fb-ea3cd560f70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:01.912758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee48ae0-a50d-4b30-83fb-ea3cd560f70d None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:01.913036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee48ae0-a50d-4b30-83fb-ea3cd560f70d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:01.913316 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee48ae0-a50d-4b30-83fb-ea3cd560f70d None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:01.913948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee48ae0-a50d-4b30-83fb-ea3cd560f70d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:01.915432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee48ae0-a50d-4b30-83fb-ea3cd560f70d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:02.017770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 599/1205] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Sep 16 07:50:01 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:50:04.178536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80b4b7-d9be-49af-a107-272fd6fb0f14 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:04.179137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80b4b7-d9be-49af-a107-272fd6fb0f14 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:04.179137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80b4b7-d9be-49af-a107-272fd6fb0f14 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:04.180496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 607/1206] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:50:04.185868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03b9eb-0933-4624-9e38-6c2342376025 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:04.186311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03b9eb-0933-4624-9e38-6c2342376025 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:04.186420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03b9eb-0933-4624-9e38-6c2342376025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:04.204657 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2b03b9eb-0933-4624-9e38-6c2342376025 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:50:04.205916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2b03b9eb-0933-4624-9e38-6c2342376025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:50:04.271171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 600/1207] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:50:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:50:04.541769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80b4b7-d9be-49af-a107-272fd6fb0f14 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:04.644043 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80b4b7-d9be-49af-a107-272fd6fb0f14 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:04.676617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86227b17-462a-441f-af36-5c65c4323138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:04.677237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86227b17-462a-441f-af36-5c65c4323138 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:04.677512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86227b17-462a-441f-af36-5c65c4323138 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:04.677778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86227b17-462a-441f-af36-5c65c4323138 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:04.688816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86227b17-462a-441f-af36-5c65c4323138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-86227b17-462a-441f-af36-5c65c4323138 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:50:04.693663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:04.694888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:04.695975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 608/1208] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:50:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:04.701137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03b9eb-0933-4624-9e38-6c2342376025 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:04.742077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:04.799905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03b9eb-0933-4624-9e38-6c2342376025 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:04.833343 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323d01f-25ba-4485-bf4d-baf166b1d4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:04.833929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323d01f-25ba-4485-bf4d-baf166b1d4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:04.834205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323d01f-25ba-4485-bf4d-baf166b1d4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:04.834472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323d01f-25ba-4485-bf4d-baf166b1d4e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:04.835156 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323d01f-25ba-4485-bf4d-baf166b1d4e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:04.835454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323d01f-25ba-4485-bf4d-baf166b1d4e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:04.837336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323d01f-25ba-4485-bf4d-baf166b1d4e0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:04.844916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 601/1209] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:50:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:04.851046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86227b17-462a-441f-af36-5c65c4323138 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:04.899559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:04.947178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86227b17-462a-441f-af36-5c65c4323138 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:04.979248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140369a4-3e27-4bba-9a13-8f9ae6ef7e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:04.979859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140369a4-3e27-4bba-9a13-8f9ae6ef7e78 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:04.980135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140369a4-3e27-4bba-9a13-8f9ae6ef7e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:04.980404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140369a4-3e27-4bba-9a13-8f9ae6ef7e78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:04.990333 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140369a4-3e27-4bba-9a13-8f9ae6ef7e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:04.991780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140369a4-3e27-4bba-9a13-8f9ae6ef7e78 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-140369a4-3e27-4bba-9a13-8f9ae6ef7e78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:05.005247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:05.006518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:05.007753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:05.007753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:05.007753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:05.007753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 609/1210] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:05.012363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323d01f-25ba-4485-bf4d-baf166b1d4e0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:05.106715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:05.109540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323d01f-25ba-4485-bf4d-baf166b1d4e0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:05.141208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deea731-f344-444d-a317-49b5b2108420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:05.141801 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deea731-f344-444d-a317-49b5b2108420 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:05.142074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deea731-f344-444d-a317-49b5b2108420 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:05.142388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deea731-f344-444d-a317-49b5b2108420 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:05.143048 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deea731-f344-444d-a317-49b5b2108420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:05.143362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deea731-f344-444d-a317-49b5b2108420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:05.144703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deea731-f344-444d-a317-49b5b2108420 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:05.156289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 602/1211] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:05.162034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140369a4-3e27-4bba-9a13-8f9ae6ef7e78 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:05.259981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140369a4-3e27-4bba-9a13-8f9ae6ef7e78 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:05.261563 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:05.292493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc396171-ff33-4124-9279-c4de982f65b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:05.293942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc396171-ff33-4124-9279-c4de982f65b0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:05.293942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc396171-ff33-4124-9279-c4de982f65b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:05.293942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc396171-ff33-4124-9279-c4de982f65b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:05.300933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc396171-ff33-4124-9279-c4de982f65b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:05.302394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc396171-ff33-4124-9279-c4de982f65b0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-bc396171-ff33-4124-9279-c4de982f65b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:50:05.313551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:05.319816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:05.321531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:05.321531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:05.321531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:50:05.321531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:50:05.321531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:50:05.321531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:50:05.321531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:50:05.321531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:50:05.321531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:05.321531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:05.321531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 610/1212] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:05.322293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deea731-f344-444d-a317-49b5b2108420 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:05.418877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deea731-f344-444d-a317-49b5b2108420 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:05.452031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d9ab8-18ae-4767-a4f6-bfe446bc163f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:05.452632 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d9ab8-18ae-4767-a4f6-bfe446bc163f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:05.452910 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d9ab8-18ae-4767-a4f6-bfe446bc163f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:05.453182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d9ab8-18ae-4767-a4f6-bfe446bc163f None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:05.453901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d9ab8-18ae-4767-a4f6-bfe446bc163f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:05.454204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d9ab8-18ae-4767-a4f6-bfe446bc163f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:05.455657 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d9ab8-18ae-4767-a4f6-bfe446bc163f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:05.472872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 603/1213] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Sep 16 07:50:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:05.478822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc396171-ff33-4124-9279-c4de982f65b0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:05.579549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc396171-ff33-4124-9279-c4de982f65b0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:05.612569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8ccd5-3309-4f70-97f3-db4b62eb943b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:05.613167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8ccd5-3309-4f70-97f3-db4b62eb943b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:05.613490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8ccd5-3309-4f70-97f3-db4b62eb943b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:05.613759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8ccd5-3309-4f70-97f3-db4b62eb943b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:05.643220 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:05.678747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8ccd5-3309-4f70-97f3-db4b62eb943b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:05.680910 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8ccd5-3309-4f70-97f3-db4b62eb943b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:05.778605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 611/1214] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:50:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:50:05.784482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d9ab8-18ae-4767-a4f6-bfe446bc163f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:05.802847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:05.880921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d9ab8-18ae-4767-a4f6-bfe446bc163f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:05.914132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48376d36-87c2-4ba1-84c0-3e258ba107ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:05.914807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48376d36-87c2-4ba1-84c0-3e258ba107ad None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:05.915085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48376d36-87c2-4ba1-84c0-3e258ba107ad None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:05.915352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48376d36-87c2-4ba1-84c0-3e258ba107ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:05.925780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48376d36-87c2-4ba1-84c0-3e258ba107ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:05.927261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48376d36-87c2-4ba1-84c0-3e258ba107ad None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-48376d36-87c2-4ba1-84c0-3e258ba107ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:05.942201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:05.943173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:05.944217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:05.944217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:05.944217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:05.944217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 604/1215] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:05.949477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8ccd5-3309-4f70-97f3-db4b62eb943b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:06.008679 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:06.045273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8ccd5-3309-4f70-97f3-db4b62eb943b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:06.078058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40198f5-85f3-4e47-9e56-c36f87c8f606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:06.078656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40198f5-85f3-4e47-9e56-c36f87c8f606 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:06.078965 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40198f5-85f3-4e47-9e56-c36f87c8f606 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:06.079236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40198f5-85f3-4e47-9e56-c36f87c8f606 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:06.079867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40198f5-85f3-4e47-9e56-c36f87c8f606 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:06.080167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40198f5-85f3-4e47-9e56-c36f87c8f606 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:06.081521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40198f5-85f3-4e47-9e56-c36f87c8f606 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:06.092191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 612/1216] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:06.097900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48376d36-87c2-4ba1-84c0-3e258ba107ad None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:06.167290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:06.197491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48376d36-87c2-4ba1-84c0-3e258ba107ad None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:06.230871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93e1b0-c7f0-4be6-8c94-40693cc01f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:06.231528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93e1b0-c7f0-4be6-8c94-40693cc01f5f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:06.231820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93e1b0-c7f0-4be6-8c94-40693cc01f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:06.232090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93e1b0-c7f0-4be6-8c94-40693cc01f5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:06.238704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93e1b0-c7f0-4be6-8c94-40693cc01f5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:06.240194 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93e1b0-c7f0-4be6-8c94-40693cc01f5f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-be93e1b0-c7f0-4be6-8c94-40693cc01f5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:50:06.251859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:06.252931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:06.253834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:50:06.253834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:50:06.253834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:50:06.253834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:50:06.253834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:50:06.253834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:50:06.253834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:06.253834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:06.253834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 605/1217] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:50:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:06.259021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40198f5-85f3-4e47-9e56-c36f87c8f606 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:06.356388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40198f5-85f3-4e47-9e56-c36f87c8f606 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:06.377558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:06.388469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d5263-77d0-43f5-abaf-44e3aaa8f2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:06.389058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d5263-77d0-43f5-abaf-44e3aaa8f2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:06.389335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d5263-77d0-43f5-abaf-44e3aaa8f2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:06.389605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d5263-77d0-43f5-abaf-44e3aaa8f2f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:06.390280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d5263-77d0-43f5-abaf-44e3aaa8f2f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:06.390581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d5263-77d0-43f5-abaf-44e3aaa8f2f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:06.392028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d5263-77d0-43f5-abaf-44e3aaa8f2f8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:06.403907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 613/1218] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:50:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:06.409919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93e1b0-c7f0-4be6-8c94-40693cc01f5f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:06.509149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93e1b0-c7f0-4be6-8c94-40693cc01f5f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:06.536776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:06.542974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca965aa4-454d-4b15-8d13-ed9d74ffe877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:06.543573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca965aa4-454d-4b15-8d13-ed9d74ffe877 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:06.543852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca965aa4-454d-4b15-8d13-ed9d74ffe877 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:06.544126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca965aa4-454d-4b15-8d13-ed9d74ffe877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:06.544848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca965aa4-454d-4b15-8d13-ed9d74ffe877 None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=c2dca05dfe8b4af0a02ea93fbe29c556) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:06.545153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca965aa4-454d-4b15-8d13-ed9d74ffe877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:06.546881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca965aa4-454d-4b15-8d13-ed9d74ffe877 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:06.552092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 606/1219] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:50:06 2022] GET /identity/v3/role_assignments?user.id=c2dca05dfe8b4af0a02ea93fbe29c556&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:08.642209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8e692-6c31-4e88-a8e7-4dbcdf4d7971 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:08.642800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8e692-6c31-4e88-a8e7-4dbcdf4d7971 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:08.642800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8e692-6c31-4e88-a8e7-4dbcdf4d7971 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:08.644195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 614/1220] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:50:08.649894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9588e-1bbb-4102-807f-fc7a2762820e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:08.650294 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9588e-1bbb-4102-807f-fc7a2762820e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:08.654565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9588e-1bbb-4102-807f-fc7a2762820e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:08.669153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-17f9588e-1bbb-4102-807f-fc7a2762820e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:50:08.670374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-17f9588e-1bbb-4102-807f-fc7a2762820e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:50:08.737785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 607/1221] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:50:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:50:09.007083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8e692-6c31-4e88-a8e7-4dbcdf4d7971 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:09.107310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8e692-6c31-4e88-a8e7-4dbcdf4d7971 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:09.140347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d2f06-d928-49a3-aab9-b6e05c50773e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:09.142972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d2f06-d928-49a3-aab9-b6e05c50773e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:09.142972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d2f06-d928-49a3-aab9-b6e05c50773e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:09.142972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d2f06-d928-49a3-aab9-b6e05c50773e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:09.152357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d2f06-d928-49a3-aab9-b6e05c50773e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-c24d2f06-d928-49a3-aab9-b6e05c50773e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:50:09.157298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:09.158427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:09.159721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 615/1222] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:50:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:09.164467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9588e-1bbb-4102-807f-fc7a2762820e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:09.262206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9588e-1bbb-4102-807f-fc7a2762820e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:09.285209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:09.296116 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dd820-7a91-4000-a43b-449be499ac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:09.296757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dd820-7a91-4000-a43b-449be499ac55 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:09.297039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dd820-7a91-4000-a43b-449be499ac55 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:09.297312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dd820-7a91-4000-a43b-449be499ac55 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:09.297999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dd820-7a91-4000-a43b-449be499ac55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:09.298302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dd820-7a91-4000-a43b-449be499ac55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:09.299707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dd820-7a91-4000-a43b-449be499ac55 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:09.307527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 608/1223] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:50:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:09.313304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d2f06-d928-49a3-aab9-b6e05c50773e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:09.358194 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:09.413181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d2f06-d928-49a3-aab9-b6e05c50773e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:09.446483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbe8f8-d60f-43a1-bd29-0e987986b18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:09.447154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe8f8-d60f-43a1-bd29-0e987986b18c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:09.447432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe8f8-d60f-43a1-bd29-0e987986b18c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:09.447700 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe8f8-d60f-43a1-bd29-0e987986b18c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:09.457389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbe8f8-d60f-43a1-bd29-0e987986b18c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:09.458868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbe8f8-d60f-43a1-bd29-0e987986b18c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-f7cbe8f8-d60f-43a1-bd29-0e987986b18c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:09.473870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:09.477773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:09.478929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:09.478929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:09.478929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:09.478929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 616/1224] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:09.480700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dd820-7a91-4000-a43b-449be499ac55 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:09.578854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dd820-7a91-4000-a43b-449be499ac55 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:09.619739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50332f7f-8f95-4545-b495-0fef64e11f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:09.620338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50332f7f-8f95-4545-b495-0fef64e11f8a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:09.620618 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50332f7f-8f95-4545-b495-0fef64e11f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:09.620890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50332f7f-8f95-4545-b495-0fef64e11f8a None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:09.621526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50332f7f-8f95-4545-b495-0fef64e11f8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:09.621825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50332f7f-8f95-4545-b495-0fef64e11f8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:09.627066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50332f7f-8f95-4545-b495-0fef64e11f8a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:09.642756 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 609/1225] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:09.648425 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbe8f8-d60f-43a1-bd29-0e987986b18c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:09.740934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:09.748198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbe8f8-d60f-43a1-bd29-0e987986b18c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:09.779620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fc1dd-d9fc-4278-965e-f7dd0973ab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:09.780232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fc1dd-d9fc-4278-965e-f7dd0973ab59 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:09.780507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fc1dd-d9fc-4278-965e-f7dd0973ab59 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:09.780775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fc1dd-d9fc-4278-965e-f7dd0973ab59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:09.787918 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fc1dd-d9fc-4278-965e-f7dd0973ab59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:09.789849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fc1dd-d9fc-4278-965e-f7dd0973ab59 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-ec0fc1dd-d9fc-4278-965e-f7dd0973ab59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:50:09.800595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:09.801720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:09.802945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:09.802945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:09.802945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:50:09.802945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:50:09.802945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:50:09.802945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:50:09.802945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:50:09.802945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:50:09.802945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:09.802945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:09.802945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 617/1226] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:09.808332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50332f7f-8f95-4545-b495-0fef64e11f8a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:09.854723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:09.905504 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50332f7f-8f95-4545-b495-0fef64e11f8a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:09.939554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71fe13-518d-4fff-8ce8-c462f44db1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:09.940155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71fe13-518d-4fff-8ce8-c462f44db1e4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:09.944890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71fe13-518d-4fff-8ce8-c462f44db1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:09.944890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71fe13-518d-4fff-8ce8-c462f44db1e4 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:09.944890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71fe13-518d-4fff-8ce8-c462f44db1e4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:09.944890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71fe13-518d-4fff-8ce8-c462f44db1e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:09.944890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71fe13-518d-4fff-8ce8-c462f44db1e4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:09.959691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 610/1227] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Sep 16 07:50:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:09.965554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fc1dd-d9fc-4278-965e-f7dd0973ab59 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:10.063845 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fc1dd-d9fc-4278-965e-f7dd0973ab59 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:10.096390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9bf65-243c-4990-b0b2-bfbd6a7a703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:10.097013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9bf65-243c-4990-b0b2-bfbd6a7a703e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:10.097287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9bf65-243c-4990-b0b2-bfbd6a7a703e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:10.097555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9bf65-243c-4990-b0b2-bfbd6a7a703e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:10.108379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9bf65-243c-4990-b0b2-bfbd6a7a703e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:10.109744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9bf65-243c-4990-b0b2-bfbd6a7a703e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-3cc9bf65-243c-4990-b0b2-bfbd6a7a703e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:10.123919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:10.125090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:10.126357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:10.126357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:10.126357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:10.126357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 618/1228] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:10.130769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71fe13-518d-4fff-8ce8-c462f44db1e4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:10.227969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71fe13-518d-4fff-8ce8-c462f44db1e4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:10.230351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:10.261103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301c160-11ca-4839-935a-d2ab5a67eda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:10.261709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301c160-11ca-4839-935a-d2ab5a67eda5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:10.262012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301c160-11ca-4839-935a-d2ab5a67eda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:10.262280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301c160-11ca-4839-935a-d2ab5a67eda5 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:10.262951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301c160-11ca-4839-935a-d2ab5a67eda5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:10.263255 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301c160-11ca-4839-935a-d2ab5a67eda5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:10.264641 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301c160-11ca-4839-935a-d2ab5a67eda5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:10.276064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 611/1229] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:10.282273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9bf65-243c-4990-b0b2-bfbd6a7a703e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:10.292401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:10.381195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9bf65-243c-4990-b0b2-bfbd6a7a703e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:10.413966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331ae22-7533-448f-84c4-0a3f447ef2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:10.415199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331ae22-7533-448f-84c4-0a3f447ef2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:10.415477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331ae22-7533-448f-84c4-0a3f447ef2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:10.415745 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331ae22-7533-448f-84c4-0a3f447ef2f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:10.422272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331ae22-7533-448f-84c4-0a3f447ef2f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:10.423718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331ae22-7533-448f-84c4-0a3f447ef2f9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-2331ae22-7533-448f-84c4-0a3f447ef2f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:50:10.435037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:10.436247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:10.437195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:50:10.437195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:50:10.437195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:50:10.437195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:50:10.437195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:50:10.437195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:50:10.437195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:10.437195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:10.437195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 619/1230] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:10.442088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301c160-11ca-4839-935a-d2ab5a67eda5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:10.540862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301c160-11ca-4839-935a-d2ab5a67eda5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:10.573960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b584e0-c139-4d21-a3a2-31a0a0c2420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:10.574620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b584e0-c139-4d21-a3a2-31a0a0c2420a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:10.574987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b584e0-c139-4d21-a3a2-31a0a0c2420a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:10.575333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b584e0-c139-4d21-a3a2-31a0a0c2420a None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:10.576030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b584e0-c139-4d21-a3a2-31a0a0c2420a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:10.576336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b584e0-c139-4d21-a3a2-31a0a0c2420a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:10.577770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b584e0-c139-4d21-a3a2-31a0a0c2420a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:10.590258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 612/1231] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:10.596368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331ae22-7533-448f-84c4-0a3f447ef2f9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:10.660971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:10.695060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331ae22-7533-448f-84c4-0a3f447ef2f9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:10.727684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516af14e-6e2f-400c-91ed-dbfa759a7eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:10.728340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516af14e-6e2f-400c-91ed-dbfa759a7eef None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:10.728614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516af14e-6e2f-400c-91ed-dbfa759a7eef None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:10.728885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516af14e-6e2f-400c-91ed-dbfa759a7eef None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/c2dca05dfe8b4af0a02ea93fbe29c556/roles/691d078c15384159b1eec2b2ceaad9ed` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:10.743289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516af14e-6e2f-400c-91ed-dbfa759a7eef None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=c2dca05dfe8b4af0a02ea93fbe29c556, role_id=691d078c15384159b1eec2b2ceaad9ed)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:10.744803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516af14e-6e2f-400c-91ed-dbfa759a7eef None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:10.851236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 620/1232] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:50:10 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/c2dca05dfe8b4af0a02ea93fbe29c556/roles/691d078c15384159b1eec2b2ceaad9ed => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:50:12.969458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95084143-fa2c-4b4d-8ceb-4c8e6e04842e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:12.970017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95084143-fa2c-4b4d-8ceb-4c8e6e04842e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:12.970017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95084143-fa2c-4b4d-8ceb-4c8e6e04842e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:12.971393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 613/1233] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:50:12.976529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77434823-9076-4338-afdb-32afa75668b1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:12.979071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77434823-9076-4338-afdb-32afa75668b1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:12.979071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77434823-9076-4338-afdb-32afa75668b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:12.995661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-77434823-9076-4338-afdb-32afa75668b1 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:50:12.996811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-77434823-9076-4338-afdb-32afa75668b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:50:13.061935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 621/1234] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:50:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:50:13.335510 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95084143-fa2c-4b4d-8ceb-4c8e6e04842e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:13.359881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:13.433650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95084143-fa2c-4b4d-8ceb-4c8e6e04842e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:13.467589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ba78c-889d-4352-b41b-f3580bb06f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:13.468225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ba78c-889d-4352-b41b-f3580bb06f92 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:13.468578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ba78c-889d-4352-b41b-f3580bb06f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:13.468849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ba78c-889d-4352-b41b-f3580bb06f92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:13.479400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ba78c-889d-4352-b41b-f3580bb06f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-247ba78c-889d-4352-b41b-f3580bb06f92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:50:13.484180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:13.485314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:13.486535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 614/1235] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Sep 16 07:50:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:13.491286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77434823-9076-4338-afdb-32afa75668b1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:13.535986 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:13.586970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77434823-9076-4338-afdb-32afa75668b1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:13.620788 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0ca04-ba59-489f-854c-062afa818b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:13.621384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0ca04-ba59-489f-854c-062afa818b8a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:13.621659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0ca04-ba59-489f-854c-062afa818b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:13.621994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0ca04-ba59-489f-854c-062afa818b8a None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:13.622648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0ca04-ba59-489f-854c-062afa818b8a None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:13.622982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0ca04-ba59-489f-854c-062afa818b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:13.624347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0ca04-ba59-489f-854c-062afa818b8a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:13.632012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 622/1236] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Sep 16 07:50:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:13.638033 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ba78c-889d-4352-b41b-f3580bb06f92 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:13.720644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:13.736220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ba78c-889d-4352-b41b-f3580bb06f92 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:13.770279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56010859-0da3-470d-a316-4eb60232c92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:13.770979 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56010859-0da3-470d-a316-4eb60232c92e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:13.771257 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56010859-0da3-470d-a316-4eb60232c92e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:13.771531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56010859-0da3-470d-a316-4eb60232c92e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:13.782520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56010859-0da3-470d-a316-4eb60232c92e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:13.783984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56010859-0da3-470d-a316-4eb60232c92e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-56010859-0da3-470d-a316-4eb60232c92e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:13.798539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:13.799730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:13.800702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:13.800702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:13.800702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:13.805418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 615/1237] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:13.806535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0ca04-ba59-489f-854c-062afa818b8a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:13.905156 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0ca04-ba59-489f-854c-062afa818b8a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:13.913603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:13.938036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dc9c3-350e-41af-a0da-ae8c3b3e6d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:13.938644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dc9c3-350e-41af-a0da-ae8c3b3e6d8a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:13.938968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dc9c3-350e-41af-a0da-ae8c3b3e6d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:13.939238 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033dc9c3-350e-41af-a0da-ae8c3b3e6d8a None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:13.939868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033dc9c3-350e-41af-a0da-ae8c3b3e6d8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:13.940172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033dc9c3-350e-41af-a0da-ae8c3b3e6d8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:13.941526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033dc9c3-350e-41af-a0da-ae8c3b3e6d8a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:13.952639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 623/1238] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:13.958376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56010859-0da3-470d-a316-4eb60232c92e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:14.061613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56010859-0da3-470d-a316-4eb60232c92e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:14.094691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed772ae5-1c7d-4a39-8f94-d62a39a26869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:14.095322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed772ae5-1c7d-4a39-8f94-d62a39a26869 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:14.095606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed772ae5-1c7d-4a39-8f94-d62a39a26869 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:14.095864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed772ae5-1c7d-4a39-8f94-d62a39a26869 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:14.098596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:14.104135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed772ae5-1c7d-4a39-8f94-d62a39a26869 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:14.105565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed772ae5-1c7d-4a39-8f94-d62a39a26869 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ed772ae5-1c7d-4a39-8f94-d62a39a26869 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:50:14.117004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:14.118152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:14.119486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:14.119486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:14.119486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:50:14.119486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:50:14.119486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:50:14.119486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:50:14.119486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:50:14.119486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:50:14.119486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:14.119486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:14.119486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 616/1239] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:14.124090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dc9c3-350e-41af-a0da-ae8c3b3e6d8a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:14.221495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033dc9c3-350e-41af-a0da-ae8c3b3e6d8a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:14.254288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862d2c0-706f-4118-b08b-0749664f6922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:14.254927 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862d2c0-706f-4118-b08b-0749664f6922 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:14.255202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862d2c0-706f-4118-b08b-0749664f6922 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:14.255467 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862d2c0-706f-4118-b08b-0749664f6922 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:14.256177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862d2c0-706f-4118-b08b-0749664f6922 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:14.256479 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862d2c0-706f-4118-b08b-0749664f6922 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:14.258463 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862d2c0-706f-4118-b08b-0749664f6922 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:14.275455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 624/1240] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Sep 16 07:50:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:14.281443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed772ae5-1c7d-4a39-8f94-d62a39a26869 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:14.378035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed772ae5-1c7d-4a39-8f94-d62a39a26869 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:14.410648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af58baa-afc1-46e9-ba16-a95023ba03c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:14.411262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af58baa-afc1-46e9-ba16-a95023ba03c9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:14.411535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af58baa-afc1-46e9-ba16-a95023ba03c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:14.411804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af58baa-afc1-46e9-ba16-a95023ba03c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:14.470099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:14.476315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af58baa-afc1-46e9-ba16-a95023ba03c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:14.477731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af58baa-afc1-46e9-ba16-a95023ba03c9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:14.573468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 617/1241] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:50:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:50:14.579283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862d2c0-706f-4118-b08b-0749664f6922 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:14.606819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:14.677141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862d2c0-706f-4118-b08b-0749664f6922 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:14.711869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654eaad3-f98a-4f04-a7c4-dd064e996f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:14.712492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654eaad3-f98a-4f04-a7c4-dd064e996f28 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:14.712771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654eaad3-f98a-4f04-a7c4-dd064e996f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:14.713045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654eaad3-f98a-4f04-a7c4-dd064e996f28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:14.723009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654eaad3-f98a-4f04-a7c4-dd064e996f28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:14.724454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654eaad3-f98a-4f04-a7c4-dd064e996f28 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-654eaad3-f98a-4f04-a7c4-dd064e996f28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:14.743204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:14.744288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:14.745394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:14.745394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:14.745394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:14.745394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 625/1242] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:14.747250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af58baa-afc1-46e9-ba16-a95023ba03c9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:14.838394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:14.844295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af58baa-afc1-46e9-ba16-a95023ba03c9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:14.876899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2adc2b-de62-4ab1-acfc-f4d3eca3bb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:14.877499 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2adc2b-de62-4ab1-acfc-f4d3eca3bb9e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:14.877779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2adc2b-de62-4ab1-acfc-f4d3eca3bb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:14.878054 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2adc2b-de62-4ab1-acfc-f4d3eca3bb9e None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:14.878716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2adc2b-de62-4ab1-acfc-f4d3eca3bb9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:14.879022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2adc2b-de62-4ab1-acfc-f4d3eca3bb9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:14.880458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2adc2b-de62-4ab1-acfc-f4d3eca3bb9e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:14.892723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 618/1243] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:14.898572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654eaad3-f98a-4f04-a7c4-dd064e996f28 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:14.974346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:14.995724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654eaad3-f98a-4f04-a7c4-dd064e996f28 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:15.027203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1ecff-c786-4698-98cc-be9e606e501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:15.027854 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1ecff-c786-4698-98cc-be9e606e501a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:15.028129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1ecff-c786-4698-98cc-be9e606e501a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:15.028396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1ecff-c786-4698-98cc-be9e606e501a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:15.034632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1ecff-c786-4698-98cc-be9e606e501a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:15.036082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1ecff-c786-4698-98cc-be9e606e501a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-53e1ecff-c786-4698-98cc-be9e606e501a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:50:15.047714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:15.050668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:15.051893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:50:15.051893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:50:15.051893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:50:15.051893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:50:15.051893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:50:15.051893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:50:15.051893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:15.051893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:15.051893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 626/1244] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:50:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:15.055709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2adc2b-de62-4ab1-acfc-f4d3eca3bb9e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:15.153666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2adc2b-de62-4ab1-acfc-f4d3eca3bb9e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:15.187154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c298d9-6d81-4959-a7d7-d6cc3cd33c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:15.187754 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c298d9-6d81-4959-a7d7-d6cc3cd33c37 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:15.188068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c298d9-6d81-4959-a7d7-d6cc3cd33c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:15.188342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c298d9-6d81-4959-a7d7-d6cc3cd33c37 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:15.189023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c298d9-6d81-4959-a7d7-d6cc3cd33c37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:15.189327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c298d9-6d81-4959-a7d7-d6cc3cd33c37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:15.190765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c298d9-6d81-4959-a7d7-d6cc3cd33c37 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:15.203701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 619/1245] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:15.209663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1ecff-c786-4698-98cc-be9e606e501a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:15.308685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1ecff-c786-4698-98cc-be9e606e501a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:15.340944 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a8c14-be19-42d4-a03f-e2a0dcced1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:15.344789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a8c14-be19-42d4-a03f-e2a0dcced1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:15.344789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a8c14-be19-42d4-a03f-e2a0dcced1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:15.344789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a8c14-be19-42d4-a03f-e2a0dcced1f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:15.344789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a8c14-be19-42d4-a03f-e2a0dcced1f6 None admin] RBAC: Adding query filter params (role.id=691d078c15384159b1eec2b2ceaad9ed, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=c2dca05dfe8b4af0a02ea93fbe29c556) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:15.344789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a8c14-be19-42d4-a03f-e2a0dcced1f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:15.344789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a8c14-be19-42d4-a03f-e2a0dcced1f6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:15.350027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 627/1246] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:50:15 2022] GET /identity/v3/role_assignments?user.id=c2dca05dfe8b4af0a02ea93fbe29c556&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=691d078c15384159b1eec2b2ceaad9ed => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:17.515887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8468-ccab-4243-9bc2-6641928e5a46 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:17.516438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8468-ccab-4243-9bc2-6641928e5a46 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:17.516438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8468-ccab-4243-9bc2-6641928e5a46 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:17.517765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 620/1247] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:50:17.523110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464d7be-a184-49f3-8355-163de6db4ca7 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:17.524047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464d7be-a184-49f3-8355-163de6db4ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:17.524270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464d7be-a184-49f3-8355-163de6db4ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:17.530519 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:17.542623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9464d7be-a184-49f3-8355-163de6db4ca7 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:50:17.543891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9464d7be-a184-49f3-8355-163de6db4ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:50:17.609432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 628/1248] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:50:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:50:17.875541 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c8468-ccab-4243-9bc2-6641928e5a46 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:17.893803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:17.976198 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c8468-ccab-4243-9bc2-6641928e5a46 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:18.009069 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb86be-4398-472d-bb0b-43b61893fea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:18.009749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb86be-4398-472d-bb0b-43b61893fea3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:18.010027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb86be-4398-472d-bb0b-43b61893fea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:18.010297 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb86be-4398-472d-bb0b-43b61893fea3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:18.020138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb86be-4398-472d-bb0b-43b61893fea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5ebb86be-4398-472d-bb0b-43b61893fea3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:50:18.025019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:18.026143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:18.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 621/1249] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:50:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:18.032055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464d7be-a184-49f3-8355-163de6db4ca7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:18.129403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464d7be-a184-49f3-8355-163de6db4ca7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:18.157355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:18.162017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24bafc-d83e-4c9c-93ec-b449397c725e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:18.162659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24bafc-d83e-4c9c-93ec-b449397c725e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:18.162993 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24bafc-d83e-4c9c-93ec-b449397c725e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:18.163265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24bafc-d83e-4c9c-93ec-b449397c725e None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:18.163908 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24bafc-d83e-4c9c-93ec-b449397c725e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:18.164205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24bafc-d83e-4c9c-93ec-b449397c725e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:18.166069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24bafc-d83e-4c9c-93ec-b449397c725e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:18.173870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 629/1250] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:50:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:18.179837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb86be-4398-472d-bb0b-43b61893fea3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:18.256345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:18.278277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb86be-4398-472d-bb0b-43b61893fea3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:18.311442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf1faa-bc8e-457e-ae56-868fba5e1f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:18.312067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf1faa-bc8e-457e-ae56-868fba5e1f07 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:18.312348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf1faa-bc8e-457e-ae56-868fba5e1f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:18.312621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf1faa-bc8e-457e-ae56-868fba5e1f07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:18.322241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf1faa-bc8e-457e-ae56-868fba5e1f07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:18.323733 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf1faa-bc8e-457e-ae56-868fba5e1f07 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9ebf1faa-bc8e-457e-ae56-868fba5e1f07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:18.338232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:18.339554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:18.340631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:18.340631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:18.340631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:18.340631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 622/1251] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:18.345159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24bafc-d83e-4c9c-93ec-b449397c725e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:18.441891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24bafc-d83e-4c9c-93ec-b449397c725e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:18.474407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280d260-3bd7-4327-a6f4-23eb5ef4540b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:18.475039 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280d260-3bd7-4327-a6f4-23eb5ef4540b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:18.475319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280d260-3bd7-4327-a6f4-23eb5ef4540b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:18.475591 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280d260-3bd7-4327-a6f4-23eb5ef4540b None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:18.476223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280d260-3bd7-4327-a6f4-23eb5ef4540b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:18.476524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280d260-3bd7-4327-a6f4-23eb5ef4540b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:18.477938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280d260-3bd7-4327-a6f4-23eb5ef4540b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:18.490451 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 630/1252] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:18.496244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf1faa-bc8e-457e-ae56-868fba5e1f07 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:18.593434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf1faa-bc8e-457e-ae56-868fba5e1f07 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:18.618625 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:18.626342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286799b6-25d1-471b-8830-d845f482b611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:18.626993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286799b6-25d1-471b-8830-d845f482b611 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:18.627275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286799b6-25d1-471b-8830-d845f482b611 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:18.627545 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286799b6-25d1-471b-8830-d845f482b611 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:18.634754 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286799b6-25d1-471b-8830-d845f482b611 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:18.636190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286799b6-25d1-471b-8830-d845f482b611 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-286799b6-25d1-471b-8830-d845f482b611 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:50:18.647036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:18.648230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:18.649409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:18.649409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:18.649409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:50:18.649409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:50:18.649409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:50:18.649409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:50:18.649409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:50:18.649409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:50:18.649409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:18.649409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:18.649409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 623/1253] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:18.654391 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280d260-3bd7-4327-a6f4-23eb5ef4540b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:18.710343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:18.753732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280d260-3bd7-4327-a6f4-23eb5ef4540b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:18.787235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c471f-cfcf-4a6e-a040-61d596032763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:18.787834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c471f-cfcf-4a6e-a040-61d596032763 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:18.788112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c471f-cfcf-4a6e-a040-61d596032763 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:18.788421 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c471f-cfcf-4a6e-a040-61d596032763 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:18.789106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c471f-cfcf-4a6e-a040-61d596032763 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:18.789411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c471f-cfcf-4a6e-a040-61d596032763 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:18.790880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c471f-cfcf-4a6e-a040-61d596032763 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:18.808197 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 631/1254] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Sep 16 07:50:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:18.813969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286799b6-25d1-471b-8830-d845f482b611 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:18.913332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286799b6-25d1-471b-8830-d845f482b611 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:18.945577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc66f16-e87b-40a7-9c69-5b3dd9c7cff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:18.946173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66f16-e87b-40a7-9c69-5b3dd9c7cff8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:18.947005 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66f16-e87b-40a7-9c69-5b3dd9c7cff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:18.947287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66f16-e87b-40a7-9c69-5b3dd9c7cff8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:18.999980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:19.012183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc66f16-e87b-40a7-9c69-5b3dd9c7cff8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:19.013637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc66f16-e87b-40a7-9c69-5b3dd9c7cff8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:19.108395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 624/1255] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:50:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:50:19.114370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c471f-cfcf-4a6e-a040-61d596032763 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:19.211078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c471f-cfcf-4a6e-a040-61d596032763 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:19.223593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:19.243931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3d2ba-2d28-4042-92a0-22174534e644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:19.244548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3d2ba-2d28-4042-92a0-22174534e644 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:19.244823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3d2ba-2d28-4042-92a0-22174534e644 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:19.245089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3d2ba-2d28-4042-92a0-22174534e644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:19.256135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3d2ba-2d28-4042-92a0-22174534e644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:19.257496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3d2ba-2d28-4042-92a0-22174534e644 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b7c3d2ba-2d28-4042-92a0-22174534e644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:19.271894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:19.273040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:19.274192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:19.274192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:19.274192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:19.274192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 632/1256] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:19.278938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc66f16-e87b-40a7-9c69-5b3dd9c7cff8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:19.364084 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:19.376647 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc66f16-e87b-40a7-9c69-5b3dd9c7cff8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:19.409347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39368aaf-5f03-4c98-8749-ffda334009ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:19.409945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39368aaf-5f03-4c98-8749-ffda334009ea None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:19.410223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39368aaf-5f03-4c98-8749-ffda334009ea None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:19.410497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39368aaf-5f03-4c98-8749-ffda334009ea None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:19.411163 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39368aaf-5f03-4c98-8749-ffda334009ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:19.411467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39368aaf-5f03-4c98-8749-ffda334009ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:19.412831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39368aaf-5f03-4c98-8749-ffda334009ea None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:19.424218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 625/1257] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:19.430002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3d2ba-2d28-4042-92a0-22174534e644 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:19.530482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3d2ba-2d28-4042-92a0-22174534e644 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:19.571260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ba643-2ee2-430d-83d0-5b8a94a5b289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:19.571889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ba643-2ee2-430d-83d0-5b8a94a5b289 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:19.572164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ba643-2ee2-430d-83d0-5b8a94a5b289 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:19.572433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ba643-2ee2-430d-83d0-5b8a94a5b289 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:19.579084 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ba643-2ee2-430d-83d0-5b8a94a5b289 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:19.580567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ba643-2ee2-430d-83d0-5b8a94a5b289 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-285ba643-2ee2-430d-83d0-5b8a94a5b289 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:50:19.591665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:19.592727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:19.593870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:50:19.593870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:50:19.593870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:50:19.593870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:50:19.593870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:50:19.593870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:50:19.593870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:19.593870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:19.593870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 633/1258] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:50:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:19.598696 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39368aaf-5f03-4c98-8749-ffda334009ea None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:19.696873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39368aaf-5f03-4c98-8749-ffda334009ea None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:19.729804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b00e0-5efa-4afc-be55-aba35b5a5abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:19.730406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b00e0-5efa-4afc-be55-aba35b5a5abf None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:19.730762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b00e0-5efa-4afc-be55-aba35b5a5abf None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:19.730978 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b00e0-5efa-4afc-be55-aba35b5a5abf None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:19.731660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b00e0-5efa-4afc-be55-aba35b5a5abf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:19.731993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b00e0-5efa-4afc-be55-aba35b5a5abf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:19.733715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b00e0-5efa-4afc-be55-aba35b5a5abf None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:19.746425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 626/1259] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:19.752632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ba643-2ee2-430d-83d0-5b8a94a5b289 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:19.777981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:19.851229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ba643-2ee2-430d-83d0-5b8a94a5b289 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:19.885191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e2fc0-3058-4176-ac41-7885907c5f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:19.885782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e2fc0-3058-4176-ac41-7885907c5f9f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:19.886058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e2fc0-3058-4176-ac41-7885907c5f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:19.886325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e2fc0-3058-4176-ac41-7885907c5f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:19.887060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e2fc0-3058-4176-ac41-7885907c5f9f None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=c2dca05dfe8b4af0a02ea93fbe29c556) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:19.887365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e2fc0-3058-4176-ac41-7885907c5f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:19.888794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e2fc0-3058-4176-ac41-7885907c5f9f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:19.893945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 634/1260] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:50:19 2022] GET /identity/v3/role_assignments?user.id=c2dca05dfe8b4af0a02ea93fbe29c556&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=62eb3e525077429aacc0ada63300802e => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:22.009776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654a724-7e54-45ec-98d1-bf4ae88bca46 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:22.010296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654a724-7e54-45ec-98d1-bf4ae88bca46 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:22.010296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654a724-7e54-45ec-98d1-bf4ae88bca46 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:22.011584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 627/1261] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:50:22.016752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf13438-4860-429d-a9d1-2bfe1afcd872 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:22.017201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf13438-4860-429d-a9d1-2bfe1afcd872 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:22.017303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf13438-4860-429d-a9d1-2bfe1afcd872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:22.036082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5bf13438-4860-429d-a9d1-2bfe1afcd872 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:50:22.037379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5bf13438-4860-429d-a9d1-2bfe1afcd872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:50:22.104252 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 635/1262] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:50:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:50:22.384725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654a724-7e54-45ec-98d1-bf4ae88bca46 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:22.395442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:22.485412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654a724-7e54-45ec-98d1-bf4ae88bca46 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:22.520507 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23439d-eab1-4a8c-a075-4e8f89da3420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:22.521151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23439d-eab1-4a8c-a075-4e8f89da3420 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:22.521431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23439d-eab1-4a8c-a075-4e8f89da3420 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:22.521755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23439d-eab1-4a8c-a075-4e8f89da3420 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:22.532245 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23439d-eab1-4a8c-a075-4e8f89da3420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-eb23439d-eab1-4a8c-a075-4e8f89da3420 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:50:22.537053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:22.538008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:22.538880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 628/1263] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:50:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:22.544387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf13438-4860-429d-a9d1-2bfe1afcd872 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:22.563031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:22.644349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf13438-4860-429d-a9d1-2bfe1afcd872 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:22.677621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fa0eb-27ac-4d9f-b9b4-0545283e13ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:22.678224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fa0eb-27ac-4d9f-b9b4-0545283e13ae None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:22.678501 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fa0eb-27ac-4d9f-b9b4-0545283e13ae None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:22.678851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fa0eb-27ac-4d9f-b9b4-0545283e13ae None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:22.679502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fa0eb-27ac-4d9f-b9b4-0545283e13ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:22.679803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fa0eb-27ac-4d9f-b9b4-0545283e13ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:22.681408 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fa0eb-27ac-4d9f-b9b4-0545283e13ae None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:22.689163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 636/1264] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:50:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:22.695095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23439d-eab1-4a8c-a075-4e8f89da3420 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:22.764614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:22.797012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23439d-eab1-4a8c-a075-4e8f89da3420 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:22.832358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ed68e-2020-4800-b7c9-b44d8638f142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:22.832986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ed68e-2020-4800-b7c9-b44d8638f142 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:22.833265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ed68e-2020-4800-b7c9-b44d8638f142 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:22.833538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ed68e-2020-4800-b7c9-b44d8638f142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:22.843710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ed68e-2020-4800-b7c9-b44d8638f142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:22.845152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ed68e-2020-4800-b7c9-b44d8638f142 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-137ed68e-2020-4800-b7c9-b44d8638f142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:22.859897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:22.865202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:22.866354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:22.866354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:22.866354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:22.866354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 629/1265] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:22.866802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fa0eb-27ac-4d9f-b9b4-0545283e13ae None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:22.945360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:22.963979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fa0eb-27ac-4d9f-b9b4-0545283e13ae None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:22.996478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8617c-7c69-4f39-b856-ca87c1c7214e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:22.997083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8617c-7c69-4f39-b856-ca87c1c7214e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:22.997360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8617c-7c69-4f39-b856-ca87c1c7214e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:22.997630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8617c-7c69-4f39-b856-ca87c1c7214e None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:22.998266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8617c-7c69-4f39-b856-ca87c1c7214e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:22.998568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8617c-7c69-4f39-b856-ca87c1c7214e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:23.000025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8617c-7c69-4f39-b856-ca87c1c7214e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:23.011940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 637/1266] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:50:23.017854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ed68e-2020-4800-b7c9-b44d8638f142 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:23.120136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ed68e-2020-4800-b7c9-b44d8638f142 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:23.139325 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:23.153008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c529a1-3c26-4181-839d-350aa780c2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:23.153621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c529a1-3c26-4181-839d-350aa780c2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:23.153902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c529a1-3c26-4181-839d-350aa780c2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:23.154175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c529a1-3c26-4181-839d-350aa780c2c1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:23.161363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c529a1-3c26-4181-839d-350aa780c2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:23.162909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c529a1-3c26-4181-839d-350aa780c2c1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d1c529a1-3c26-4181-839d-350aa780c2c1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:50:23.174337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:23.175516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:23.176775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:23.176775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:23.176775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:50:23.176775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:50:23.176775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:50:23.176775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:50:23.176775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:50:23.176775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:50:23.176775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:23.176775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:23.176775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 630/1267] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:23.181670 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8617c-7c69-4f39-b856-ca87c1c7214e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:23.281278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8617c-7c69-4f39-b856-ca87c1c7214e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:23.316776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50f18-6c8b-4b8c-81e9-a03adf12dda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:23.317387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50f18-6c8b-4b8c-81e9-a03adf12dda4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:23.317668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50f18-6c8b-4b8c-81e9-a03adf12dda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:23.317950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50f18-6c8b-4b8c-81e9-a03adf12dda4 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:23.318689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50f18-6c8b-4b8c-81e9-a03adf12dda4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:23.319098 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50f18-6c8b-4b8c-81e9-a03adf12dda4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:23.320512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50f18-6c8b-4b8c-81e9-a03adf12dda4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:23.329906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:23.338989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 638/1268] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Sep 16 07:50:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:23.344784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c529a1-3c26-4181-839d-350aa780c2c1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:23.442341 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c529a1-3c26-4181-839d-350aa780c2c1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:23.476494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0456b5d-aca0-45f1-aae8-6fb5aea3b82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:23.477123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0456b5d-aca0-45f1-aae8-6fb5aea3b82c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:23.477403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0456b5d-aca0-45f1-aae8-6fb5aea3b82c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:23.477673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0456b5d-aca0-45f1-aae8-6fb5aea3b82c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:23.487865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0456b5d-aca0-45f1-aae8-6fb5aea3b82c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:23.489309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0456b5d-aca0-45f1-aae8-6fb5aea3b82c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f0456b5d-aca0-45f1-aae8-6fb5aea3b82c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:23.502984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:23.504127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:23.505419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:23.505419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:23.505419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:23.505419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 631/1269] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:23.510327 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50f18-6c8b-4b8c-81e9-a03adf12dda4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:23.608611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50f18-6c8b-4b8c-81e9-a03adf12dda4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:23.642906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfeefa-f325-4bd7-adfd-bfe69a9f9bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:23.643517 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfeefa-f325-4bd7-adfd-bfe69a9f9bab None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:23.643798 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfeefa-f325-4bd7-adfd-bfe69a9f9bab None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:23.644075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfeefa-f325-4bd7-adfd-bfe69a9f9bab None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:23.644724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfeefa-f325-4bd7-adfd-bfe69a9f9bab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:23.645031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfeefa-f325-4bd7-adfd-bfe69a9f9bab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:23.646429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfeefa-f325-4bd7-adfd-bfe69a9f9bab None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:23.657605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 639/1270] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:23.663417 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0456b5d-aca0-45f1-aae8-6fb5aea3b82c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:23.694663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:23.762266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0456b5d-aca0-45f1-aae8-6fb5aea3b82c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:23.794634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c835da-01ad-46cc-bdd1-41a0aab6136e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:23.795326 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c835da-01ad-46cc-bdd1-41a0aab6136e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:23.795604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c835da-01ad-46cc-bdd1-41a0aab6136e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:23.795874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c835da-01ad-46cc-bdd1-41a0aab6136e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:23.802432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c835da-01ad-46cc-bdd1-41a0aab6136e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:23.803952 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c835da-01ad-46cc-bdd1-41a0aab6136e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-21c835da-01ad-46cc-bdd1-41a0aab6136e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:50:23.815795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:23.816781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:23.817773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:50:23.817773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:50:23.817773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:50:23.817773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:50:23.817773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:50:23.817773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:50:23.817773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:23.817773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:23.817773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 632/1271] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:50:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:23.823554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfeefa-f325-4bd7-adfd-bfe69a9f9bab None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:23.881412 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:23.921265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfeefa-f325-4bd7-adfd-bfe69a9f9bab None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:23.953717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f66e16-851c-4d29-a3d3-46c0e48d7fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:23.954313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f66e16-851c-4d29-a3d3-46c0e48d7fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:23.954595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f66e16-851c-4d29-a3d3-46c0e48d7fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:23.954893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f66e16-851c-4d29-a3d3-46c0e48d7fe0 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:23.955576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f66e16-851c-4d29-a3d3-46c0e48d7fe0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:23.955880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f66e16-851c-4d29-a3d3-46c0e48d7fe0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:23.957294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f66e16-851c-4d29-a3d3-46c0e48d7fe0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:23.968740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 640/1272] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:50:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 07:50:23.974981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c835da-01ad-46cc-bdd1-41a0aab6136e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:24.066443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:24.075007 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c835da-01ad-46cc-bdd1-41a0aab6136e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:24.107945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b534adb-af98-4574-a4f2-98fd87891576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:24.108598 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b534adb-af98-4574-a4f2-98fd87891576 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:24.108881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b534adb-af98-4574-a4f2-98fd87891576 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:24.109153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b534adb-af98-4574-a4f2-98fd87891576 None admin] PATH_INFO: `/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/c2dca05dfe8b4af0a02ea93fbe29c556/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:24.111091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b534adb-af98-4574-a4f2-98fd87891576 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7731b013b1c473fae443eb4dec76bc6, user_id=c2dca05dfe8b4af0a02ea93fbe29c556, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:24.112522 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b534adb-af98-4574-a4f2-98fd87891576 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:24.193943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 633/1273] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Sep 16 07:50:23 2022] PUT /identity/v3/projects/c7731b013b1c473fae443eb4dec76bc6/users/c2dca05dfe8b4af0a02ea93fbe29c556/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 07:50:26.303663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc8dd9-6aa7-48c5-84b6-0b0bb77fa9d4 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:26.304218 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc8dd9-6aa7-48c5-84b6-0b0bb77fa9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:26.304218 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc8dd9-6aa7-48c5-84b6-0b0bb77fa9d4 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:26.305546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 641/1274] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:50:26.311310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a767a5d-675d-4629-9789-fa2f5d0a4241 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:26.311707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a767a5d-675d-4629-9789-fa2f5d0a4241 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:26.311873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a767a5d-675d-4629-9789-fa2f5d0a4241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:26.330405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7a767a5d-675d-4629-9789-fa2f5d0a4241 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:50:26.331678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7a767a5d-675d-4629-9789-fa2f5d0a4241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:50:26.397346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 634/1275] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:50:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:50:26.665632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc8dd9-6aa7-48c5-84b6-0b0bb77fa9d4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:26.777846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc8dd9-6aa7-48c5-84b6-0b0bb77fa9d4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:26.796671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:26.815671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0873b83-200a-43ee-b392-196d697ae833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:26.816332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873b83-200a-43ee-b392-196d697ae833 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:26.820060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873b83-200a-43ee-b392-196d697ae833 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:26.820060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873b83-200a-43ee-b392-196d697ae833 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:26.829179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0873b83-200a-43ee-b392-196d697ae833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-e0873b83-200a-43ee-b392-196d697ae833 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 07:50:26.834153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:26.835280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:26.836221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 642/1276] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Sep 16 07:50:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:26.842037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a767a5d-675d-4629-9789-fa2f5d0a4241 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:26.925897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:26.944165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a767a5d-675d-4629-9789-fa2f5d0a4241 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:26.979071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343118ae-76dc-4744-b195-8ee617953ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:26.979697 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343118ae-76dc-4744-b195-8ee617953ccf None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:26.979978 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343118ae-76dc-4744-b195-8ee617953ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:26.980253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343118ae-76dc-4744-b195-8ee617953ccf None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:26.980931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343118ae-76dc-4744-b195-8ee617953ccf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:26.981241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343118ae-76dc-4744-b195-8ee617953ccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:26.982656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343118ae-76dc-4744-b195-8ee617953ccf None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:26.991291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 635/1277] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Sep 16 07:50:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:26.997751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0873b83-200a-43ee-b392-196d697ae833 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:27.096077 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0873b83-200a-43ee-b392-196d697ae833 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:27.130055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e3d3a-9cb5-4d7a-bc8d-3ec3d91d8b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:27.130701 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e3d3a-9cb5-4d7a-bc8d-3ec3d91d8b61 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:27.131070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e3d3a-9cb5-4d7a-bc8d-3ec3d91d8b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:27.131351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e3d3a-9cb5-4d7a-bc8d-3ec3d91d8b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:27.141085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e3d3a-9cb5-4d7a-bc8d-3ec3d91d8b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:27.142508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e3d3a-9cb5-4d7a-bc8d-3ec3d91d8b61 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-736e3d3a-9cb5-4d7a-bc8d-3ec3d91d8b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:27.156681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:27.157846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:27.158956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:27.158956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:27.158956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:27.158956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 643/1278] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:27.163694 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343118ae-76dc-4744-b195-8ee617953ccf None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:27.262421 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343118ae-76dc-4744-b195-8ee617953ccf None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:27.296071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ccb60-30b8-408e-9a2d-030becf05275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:27.296773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ccb60-30b8-408e-9a2d-030becf05275 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:27.297053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ccb60-30b8-408e-9a2d-030becf05275 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:27.297323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ccb60-30b8-408e-9a2d-030becf05275 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:27.297954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ccb60-30b8-408e-9a2d-030becf05275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:27.298254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ccb60-30b8-408e-9a2d-030becf05275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:27.299760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ccb60-30b8-408e-9a2d-030becf05275 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:27.312309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 636/1279] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:27.318171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e3d3a-9cb5-4d7a-bc8d-3ec3d91d8b61 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:27.349251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:27.417286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e3d3a-9cb5-4d7a-bc8d-3ec3d91d8b61 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:27.449777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a896d1-6666-46b4-a576-bb69c49b6507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:27.450469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a896d1-6666-46b4-a576-bb69c49b6507 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:27.450776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a896d1-6666-46b4-a576-bb69c49b6507 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:27.451052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a896d1-6666-46b4-a576-bb69c49b6507 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:27.458172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a896d1-6666-46b4-a576-bb69c49b6507 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:27.459634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a896d1-6666-46b4-a576-bb69c49b6507 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-e7a896d1-6666-46b4-a576-bb69c49b6507 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 07:50:27.471131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:27.477011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:27.481781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:27.481781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:27.481781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 07:50:27.481781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 07:50:27.481781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 07:50:27.481781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 07:50:27.481781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 07:50:27.481781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 07:50:27.481781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 07:50:27.481781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:27.481781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 644/1280] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:27.482574 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ccb60-30b8-408e-9a2d-030becf05275 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:27.482855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:27.573990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ccb60-30b8-408e-9a2d-030becf05275 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:27.608120 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e42ad-5fdb-469f-b797-b571802384d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:27.608723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e42ad-5fdb-469f-b797-b571802384d2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:27.609009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e42ad-5fdb-469f-b797-b571802384d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:27.609263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e42ad-5fdb-469f-b797-b571802384d2 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:27.609984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e42ad-5fdb-469f-b797-b571802384d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:27.610285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e42ad-5fdb-469f-b797-b571802384d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:27.611734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e42ad-5fdb-469f-b797-b571802384d2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:27.628290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 637/1281] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Sep 16 07:50:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:27.634152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a896d1-6666-46b4-a576-bb69c49b6507 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:27.724704 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:27.734305 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a896d1-6666-46b4-a576-bb69c49b6507 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:27.767001 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f509c-efc6-4849-b2ab-295f008d282b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:27.767631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f509c-efc6-4849-b2ab-295f008d282b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:27.767914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f509c-efc6-4849-b2ab-295f008d282b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:27.768190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f509c-efc6-4849-b2ab-295f008d282b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:27.834769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f509c-efc6-4849-b2ab-295f008d282b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:27.836255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f509c-efc6-4849-b2ab-295f008d282b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:27.921997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:27.932461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 645/1282] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Sep 16 07:50:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 07:50:27.938549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e42ad-5fdb-469f-b797-b571802384d2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:27.998599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:28.035734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e42ad-5fdb-469f-b797-b571802384d2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:28.068276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21244-2de5-4544-9b2d-ad8357edf9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:28.068896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21244-2de5-4544-9b2d-ad8357edf9ec None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:28.069172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21244-2de5-4544-9b2d-ad8357edf9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:28.069444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21244-2de5-4544-9b2d-ad8357edf9ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:28.079641 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b21244-2de5-4544-9b2d-ad8357edf9ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:28.081038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b21244-2de5-4544-9b2d-ad8357edf9ec None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-93b21244-2de5-4544-9b2d-ad8357edf9ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 07:50:28.116990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:28.118658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 07:50:28.119599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 07:50:28.119599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 07:50:28.119599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:28.119599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 638/1283] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:50:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:28.119840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f509c-efc6-4849-b2ab-295f008d282b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:28.204249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f509c-efc6-4849-b2ab-295f008d282b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:28.240591 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28669c-e36a-4f53-b764-dd5eb494ae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:28.241211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28669c-e36a-4f53-b764-dd5eb494ae80 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:28.241483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28669c-e36a-4f53-b764-dd5eb494ae80 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:28.241753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28669c-e36a-4f53-b764-dd5eb494ae80 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:28.242423 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28669c-e36a-4f53-b764-dd5eb494ae80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:28.242753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28669c-e36a-4f53-b764-dd5eb494ae80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:28.244685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28669c-e36a-4f53-b764-dd5eb494ae80 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:28.256359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 646/1284] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Sep 16 07:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:28.263244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21244-2de5-4544-9b2d-ad8357edf9ec None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:28.363602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21244-2de5-4544-9b2d-ad8357edf9ec None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:28.377551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:28.397724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe172c-1aae-4305-8cae-6f77d9d400a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:28.398352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe172c-1aae-4305-8cae-6f77d9d400a2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:28.398631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe172c-1aae-4305-8cae-6f77d9d400a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:28.398937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe172c-1aae-4305-8cae-6f77d9d400a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:28.405476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe172c-1aae-4305-8cae-6f77d9d400a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:28.406954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe172c-1aae-4305-8cae-6f77d9d400a2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d6fe172c-1aae-4305-8cae-6f77d9d400a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 07:50:28.418076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:28.419075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:28.420310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 07:50:28.420310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 07:50:28.420310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 07:50:28.420310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 07:50:28.420310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 07:50:28.420310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 07:50:28.420310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 07:50:28.420310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 16 07:50:28.420310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 639/1285] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Sep 16 07:50:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 07:50:28.425190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28669c-e36a-4f53-b764-dd5eb494ae80 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:28.474563 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:28.522954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28669c-e36a-4f53-b764-dd5eb494ae80 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:28.556772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be29b43-c673-4979-a74b-099f99904a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:28.557472 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be29b43-c673-4979-a74b-099f99904a1f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:28.557749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be29b43-c673-4979-a74b-099f99904a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:28.558020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be29b43-c673-4979-a74b-099f99904a1f None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:28.558720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be29b43-c673-4979-a74b-099f99904a1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:28.559024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be29b43-c673-4979-a74b-099f99904a1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:28.560432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be29b43-c673-4979-a74b-099f99904a1f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:28.572181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 647/1286] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Sep 16 07:50:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:28.578345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe172c-1aae-4305-8cae-6f77d9d400a2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:28.677246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe172c-1aae-4305-8cae-6f77d9d400a2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:28.710911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec758b6d-75b5-43f5-b3a7-082aacb2d1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:28.711555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec758b6d-75b5-43f5-b3a7-082aacb2d1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:28.711840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec758b6d-75b5-43f5-b3a7-082aacb2d1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:28.712112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec758b6d-75b5-43f5-b3a7-082aacb2d1a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:28.712832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec758b6d-75b5-43f5-b3a7-082aacb2d1a0 None admin] RBAC: Adding query filter params (role.id=62eb3e525077429aacc0ada63300802e, scope.project.id=c7731b013b1c473fae443eb4dec76bc6, user.id=c2dca05dfe8b4af0a02ea93fbe29c556) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:28.713138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec758b6d-75b5-43f5-b3a7-082aacb2d1a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:28.714576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec758b6d-75b5-43f5-b3a7-082aacb2d1a0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:28.720383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 640/1287] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Sep 16 07:50:28 2022] GET /identity/v3/role_assignments?user.id=c2dca05dfe8b4af0a02ea93fbe29c556&scope.project.id=c7731b013b1c473fae443eb4dec76bc6&role.id=62eb3e525077429aacc0ada63300802e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:30.900851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a2526-2f0c-4586-bcbe-46f3d2006b39 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:30.901429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a2526-2f0c-4586-bcbe-46f3d2006b39 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:30.901429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a2526-2f0c-4586-bcbe-46f3d2006b39 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:30.902744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 648/1288] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:50:30.908016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436db7-d646-4780-b186-6cc01f256a51 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:30.908478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436db7-d646-4780-b186-6cc01f256a51 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:30.908621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436db7-d646-4780-b186-6cc01f256a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:30.926993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-71436db7-d646-4780-b186-6cc01f256a51 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:50:30.928177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-71436db7-d646-4780-b186-6cc01f256a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:50:30.957531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:30.993917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 641/1289] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:50:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:50:31.263233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a2526-2f0c-4586-bcbe-46f3d2006b39 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:31.363161 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a2526-2f0c-4586-bcbe-46f3d2006b39 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:31.373466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:31.395785 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36ae84-abfc-4f6d-9d32-31c7e2445f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:31.396414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36ae84-abfc-4f6d-9d32-31c7e2445f02 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:31.396690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36ae84-abfc-4f6d-9d32-31c7e2445f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:31.396982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36ae84-abfc-4f6d-9d32-31c7e2445f02 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:31.397561 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36ae84-abfc-4f6d-9d32-31c7e2445f02 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:31.399442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36ae84-abfc-4f6d-9d32-31c7e2445f02 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-0f36ae84-abfc-4f6d-9d32-31c7e2445f02 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:31.408112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:31.409399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:31.410520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 649/1290] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Sep 16 07:50:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:31.415810 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71436db7-d646-4780-b186-6cc01f256a51 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:31.512087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71436db7-d646-4780-b186-6cc01f256a51 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:31.545995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bdeb9-2372-46ad-89de-4ed153c73b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:31.546543 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bdeb9-2372-46ad-89de-4ed153c73b64 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:31.546851 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bdeb9-2372-46ad-89de-4ed153c73b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:31.547128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bdeb9-2372-46ad-89de-4ed153c73b64 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:31.547782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bdeb9-2372-46ad-89de-4ed153c73b64 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:31.548082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bdeb9-2372-46ad-89de-4ed153c73b64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:31.549531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bdeb9-2372-46ad-89de-4ed153c73b64 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:31.554196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 642/1291] 10.222.0.59 () {56 vars in 1116 bytes} [Fri Sep 16 07:50:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:31.560225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36ae84-abfc-4f6d-9d32-31c7e2445f02 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:31.659519 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36ae84-abfc-4f6d-9d32-31c7e2445f02 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:31.692134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958a185-988a-473b-a41e-bcbef53a8022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:31.692733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958a185-988a-473b-a41e-bcbef53a8022 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:31.693033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958a185-988a-473b-a41e-bcbef53a8022 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:31.693309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958a185-988a-473b-a41e-bcbef53a8022 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:31.693935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958a185-988a-473b-a41e-bcbef53a8022 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:31.694240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958a185-988a-473b-a41e-bcbef53a8022 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:31.695622 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958a185-988a-473b-a41e-bcbef53a8022 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:31.700415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 650/1292] 10.222.0.59 () {56 vars in 1116 bytes} [Fri Sep 16 07:50:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:33.811018 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185ce6d-4cd4-49ea-88f8-b730dfeae8e2 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:33.811555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185ce6d-4cd4-49ea-88f8-b730dfeae8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:33.811555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185ce6d-4cd4-49ea-88f8-b730dfeae8e2 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:33.812918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 643/1293] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:50:33.818026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427861c-8140-439b-adda-3d82bcb292da None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:33.818479 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427861c-8140-439b-adda-3d82bcb292da None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:33.818611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427861c-8140-439b-adda-3d82bcb292da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:33.836898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7427861c-8140-439b-adda-3d82bcb292da None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:50:33.840861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7427861c-8140-439b-adda-3d82bcb292da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:50:33.867521 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:33.902614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 651/1294] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:50:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:50:34.168166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185ce6d-4cd4-49ea-88f8-b730dfeae8e2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:34.198052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:34.266900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185ce6d-4cd4-49ea-88f8-b730dfeae8e2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:34.300294 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce624ffc-074c-49e0-aa5f-38f96a897b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:34.300877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce624ffc-074c-49e0-aa5f-38f96a897b2d None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:34.301155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce624ffc-074c-49e0-aa5f-38f96a897b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:34.301537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce624ffc-074c-49e0-aa5f-38f96a897b2d None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:34.302192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce624ffc-074c-49e0-aa5f-38f96a897b2d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:34.303578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce624ffc-074c-49e0-aa5f-38f96a897b2d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:34.365425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 644/1295] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Sep 16 07:50:34 2022] POST /identity/v3/services => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:50:36.490198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e9d1f-8e83-4a87-8b43-f94365ae6542 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:36.490817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e9d1f-8e83-4a87-8b43-f94365ae6542 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:36.490817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e9d1f-8e83-4a87-8b43-f94365ae6542 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:36.492112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 652/1296] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:50:36.497384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36444c41-8165-4517-b810-49e6a24ed0ac None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:36.497826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36444c41-8165-4517-b810-49e6a24ed0ac None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:36.497935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36444c41-8165-4517-b810-49e6a24ed0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:36.515890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-36444c41-8165-4517-b810-49e6a24ed0ac None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:50:36.517127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-36444c41-8165-4517-b810-49e6a24ed0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:50:36.588457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 645/1297] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:50:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:50:36.857099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e9d1f-8e83-4a87-8b43-f94365ae6542 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:36.956273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e9d1f-8e83-4a87-8b43-f94365ae6542 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:36.989219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c93b28-80a5-40ef-800a-7cc4e394cdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:36.989841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c93b28-80a5-40ef-800a-7cc4e394cdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:36.990118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c93b28-80a5-40ef-800a-7cc4e394cdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:36.990390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c93b28-80a5-40ef-800a-7cc4e394cdf2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:36.991040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c93b28-80a5-40ef-800a-7cc4e394cdf2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:36.992398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c93b28-80a5-40ef-800a-7cc4e394cdf2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-85c93b28-80a5-40ef-800a-7cc4e394cdf2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:37.000872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:37.002089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:37.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 653/1298] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Sep 16 07:50:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:37.008158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36444c41-8165-4517-b810-49e6a24ed0ac None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:37.027033 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:37.104525 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36444c41-8165-4517-b810-49e6a24ed0ac None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:37.138530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acc95b-9ae1-455c-a8ab-c220d7024bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:37.139147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acc95b-9ae1-455c-a8ab-c220d7024bec None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:37.142818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acc95b-9ae1-455c-a8ab-c220d7024bec None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:37.142818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acc95b-9ae1-455c-a8ab-c220d7024bec None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:37.142818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acc95b-9ae1-455c-a8ab-c220d7024bec None admin] RBAC: Adding query filter params (name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:37.142818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acc95b-9ae1-455c-a8ab-c220d7024bec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:37.142818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acc95b-9ae1-455c-a8ab-c220d7024bec None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:37.147197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 646/1299] 10.222.0.59 () {56 vars in 1116 bytes} [Fri Sep 16 07:50:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:37.153228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c93b28-80a5-40ef-800a-7cc4e394cdf2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:37.185019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:37.250469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c93b28-80a5-40ef-800a-7cc4e394cdf2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:37.283994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae1442-5dfc-4a40-b36d-1338a58812d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:37.284596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae1442-5dfc-4a40-b36d-1338a58812d3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:37.284876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae1442-5dfc-4a40-b36d-1338a58812d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:37.285179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae1442-5dfc-4a40-b36d-1338a58812d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:37.285873 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae1442-5dfc-4a40-b36d-1338a58812d3 None admin] RBAC: Adding query filter params (interface=public, service_id=fa1c038125ef4dd3bc8192439f4bc960, region_id=RegionOne) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:37.286214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae1442-5dfc-4a40-b36d-1338a58812d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:37.287600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae1442-5dfc-4a40-b36d-1338a58812d3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:37.293137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 654/1300] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Sep 16 07:50:37 2022] GET /identity/v3/endpoints?service_id=fa1c038125ef4dd3bc8192439f4bc960&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:37.298752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acc95b-9ae1-455c-a8ab-c220d7024bec None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:37.382544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:37.402707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acc95b-9ae1-455c-a8ab-c220d7024bec None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:37.436403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3de5b-e353-4955-9dc6-0f7d288c33c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:37.437002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3de5b-e353-4955-9dc6-0f7d288c33c9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:37.437284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3de5b-e353-4955-9dc6-0f7d288c33c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:37.437565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3de5b-e353-4955-9dc6-0f7d288c33c9 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:37.438159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3de5b-e353-4955-9dc6-0f7d288c33c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:37.439551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3de5b-e353-4955-9dc6-0f7d288c33c9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:37.446629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 647/1301] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Sep 16 07:50:37 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 07:50:39.582472 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246794de-2a29-4e32-a492-a0be8b0cbfdf None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:39.583060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246794de-2a29-4e32-a492-a0be8b0cbfdf None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:39.583060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246794de-2a29-4e32-a492-a0be8b0cbfdf None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:39.584396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 655/1302] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:50:39.590342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0809521-f8fc-4b07-95c2-7ee7824f1b44 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:39.590784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0809521-f8fc-4b07-95c2-7ee7824f1b44 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:39.591956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0809521-f8fc-4b07-95c2-7ee7824f1b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:39.609317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d0809521-f8fc-4b07-95c2-7ee7824f1b44 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:50:39.610534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d0809521-f8fc-4b07-95c2-7ee7824f1b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:50:39.674901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 648/1303] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:50:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 07:50:39.941075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246794de-2a29-4e32-a492-a0be8b0cbfdf None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:40.033765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:40.040049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246794de-2a29-4e32-a492-a0be8b0cbfdf None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:40.073040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952050f7-a92e-4dee-8bcb-043ae6cf452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:40.073655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952050f7-a92e-4dee-8bcb-043ae6cf452f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:40.073930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952050f7-a92e-4dee-8bcb-043ae6cf452f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:40.074198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952050f7-a92e-4dee-8bcb-043ae6cf452f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:40.074796 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952050f7-a92e-4dee-8bcb-043ae6cf452f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:40.076182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952050f7-a92e-4dee-8bcb-043ae6cf452f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-952050f7-a92e-4dee-8bcb-043ae6cf452f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 07:50:40.085181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 07:50:40.086358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 16 07:50:40.087450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 656/1304] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Sep 16 07:50:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 07:50:40.092305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0809521-f8fc-4b07-95c2-7ee7824f1b44 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:40.166695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:40.189153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0809521-f8fc-4b07-95c2-7ee7824f1b44 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:40.221829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052909e4-1a06-4c12-86a4-b14b5b17eb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:40.222393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052909e4-1a06-4c12-86a4-b14b5b17eb38 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:40.222655 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052909e4-1a06-4c12-86a4-b14b5b17eb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:40.222959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052909e4-1a06-4c12-86a4-b14b5b17eb38 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:40.223587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052909e4-1a06-4c12-86a4-b14b5b17eb38 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 07:50:40.223889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052909e4-1a06-4c12-86a4-b14b5b17eb38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:40.225290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052909e4-1a06-4c12-86a4-b14b5b17eb38 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:40.230290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 649/1305] 10.222.0.59 () {56 vars in 1116 bytes} [Fri Sep 16 07:50:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 07:50:40.236020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952050f7-a92e-4dee-8bcb-043ae6cf452f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:50:40.333965 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952050f7-a92e-4dee-8bcb-043ae6cf452f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:50:40.365358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3534d09-0154-4960-a303-e5d25ea277e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:50:40.365947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3534d09-0154-4960-a303-e5d25ea277e9 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:50:40.366225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3534d09-0154-4960-a303-e5d25ea277e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:50:40.366495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3534d09-0154-4960-a303-e5d25ea277e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:50:40.367209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3534d09-0154-4960-a303-e5d25ea277e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:50:40.369147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3534d09-0154-4960-a303-e5d25ea277e9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:50:40.460177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 657/1306] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 07:50:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 07:50:50.248883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=30 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:50:50.455868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:41.571364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f4ed2-5869-4e41-b338-5d7f48b06b10 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:41.572357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f4ed2-5869-4e41-b338-5d7f48b06b10 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:41.572542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f4ed2-5869-4e41-b338-5d7f48b06b10 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:41.579094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8631d2-d0c4-4f50-98f3-ead25abf4953 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:41.579513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8631d2-d0c4-4f50-98f3-ead25abf4953 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:41.579663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8631d2-d0c4-4f50-98f3-ead25abf4953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:41.582472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 650/1307] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:56:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:56:41.692471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1d8631d2-d0c4-4f50-98f3-ead25abf4953 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:56:41.695465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1d8631d2-d0c4-4f50-98f3-ead25abf4953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:41.898832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 658/1308] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 07:56:42.182966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55652ef2-726e-40ee-90e4-f8025db516d2 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:42.183619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55652ef2-726e-40ee-90e4-f8025db516d2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:42.184128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55652ef2-726e-40ee-90e4-f8025db516d2 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:42.185774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 651/1309] 10.222.0.59 () {54 vars in 965 bytes} [Fri Sep 16 07:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:56:42.192189 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c985e7-0791-4a08-8c2e-a507101beaae None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:42.192504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c985e7-0791-4a08-8c2e-a507101beaae None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:42.192753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c985e7-0791-4a08-8c2e-a507101beaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:42.289809 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b7c985e7-0791-4a08-8c2e-a507101beaae None None] MFA Rules not processed for user `a29c4e53ac9a438e9701a70f3de3f32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:56:42.298930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b7c985e7-0791-4a08-8c2e-a507101beaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:42.326152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:42.390763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 659/1310] 10.222.0.59 () {58 vars in 1062 bytes} [Fri Sep 16 07:56:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:56:42.399622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55652ef2-726e-40ee-90e4-f8025db516d2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:56:42.401452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-55652ef2-726e-40ee-90e4-f8025db516d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:42.529048 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55652ef2-726e-40ee-90e4-f8025db516d2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:56:42.625357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e1874-d017-4f48-b60c-55a4aefdc3c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:56:42.626854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1874-d017-4f48-b60c-55a4aefdc3c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:42.627283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1874-d017-4f48-b60c-55a4aefdc3c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:42.627850 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1874-d017-4f48-b60c-55a4aefdc3c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:42.629731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0a2e1874-d017-4f48-b60c-55a4aefdc3c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:42.697036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:42.762913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e1874-d017-4f48-b60c-55a4aefdc3c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:56:42.764370 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e1874-d017-4f48-b60c-55a4aefdc3c3 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:56:42.796145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 652/1311] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:56:45.249782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87f41-1559-491a-bee5-fbed762becef None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:45.250266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87f41-1559-491a-bee5-fbed762becef None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:45.250359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87f41-1559-491a-bee5-fbed762becef None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:45.251768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 660/1312] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:56:45.257169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a515d99-41d8-428b-8b6c-406ce0a6635c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:45.257583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a515d99-41d8-428b-8b6c-406ce0a6635c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:45.257718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a515d99-41d8-428b-8b6c-406ce0a6635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:45.327915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0a515d99-41d8-428b-8b6c-406ce0a6635c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:56:45.329401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0a515d99-41d8-428b-8b6c-406ce0a6635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:45.421906 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 653/1313] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:56:45.705219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd87f41-1559-491a-bee5-fbed762becef None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:56:45.706826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ebd87f41-1559-491a-bee5-fbed762becef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:45.812473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd87f41-1559-491a-bee5-fbed762becef None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:56:45.880993 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:45.883247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e1f3d-9da4-4701-b12f-83e706ac98d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:56:45.883846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1f3d-9da4-4701-b12f-83e706ac98d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:45.884145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1f3d-9da4-4701-b12f-83e706ac98d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:45.884418 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1f3d-9da4-4701-b12f-83e706ac98d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:45.885996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-032e1f3d-9da4-4701-b12f-83e706ac98d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:45.992990 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e1f3d-9da4-4701-b12f-83e706ac98d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:56:45.994392 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e1f3d-9da4-4701-b12f-83e706ac98d8 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:56:46.012196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 661/1314] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:56:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:56:48.664652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b64437-29ca-496c-a075-596ab21a00af None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:48.665196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b64437-29ca-496c-a075-596ab21a00af None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:48.665196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b64437-29ca-496c-a075-596ab21a00af None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:48.666562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 654/1315] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:56:48.671837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e225fa1-0e53-4214-aa83-86691bc63ea6 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:48.672266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e225fa1-0e53-4214-aa83-86691bc63ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:48.672400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e225fa1-0e53-4214-aa83-86691bc63ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:48.739018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4e225fa1-0e53-4214-aa83-86691bc63ea6 None None] MFA Rules not processed for user `71f957f6c8724676af13c99ddc4b62e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:56:48.740234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4e225fa1-0e53-4214-aa83-86691bc63ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:48.792690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:48.836587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 662/1316] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:56:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:56:49.119017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b64437-29ca-496c-a075-596ab21a00af None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:56:49.120561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e2b64437-29ca-496c-a075-596ab21a00af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:49.130215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:49.222927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b64437-29ca-496c-a075-596ab21a00af None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:56:49.295580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc65e10-d29e-4d18-b236-00ce39649985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:56:49.296178 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc65e10-d29e-4d18-b236-00ce39649985 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:49.296453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc65e10-d29e-4d18-b236-00ce39649985 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:49.296723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc65e10-d29e-4d18-b236-00ce39649985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:49.298296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dcc65e10-d29e-4d18-b236-00ce39649985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:49.358530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:49.402566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc65e10-d29e-4d18-b236-00ce39649985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:56:49.404567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc65e10-d29e-4d18-b236-00ce39649985 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:56:49.422820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 655/1317] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:56:49 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:56:52.289376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e615-1a9b-4742-8ec6-18b1030194fe None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:52.290002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e615-1a9b-4742-8ec6-18b1030194fe None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:52.290002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e615-1a9b-4742-8ec6-18b1030194fe None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:52.291376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 663/1318] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:56:52.296498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeeb26-e0bb-472b-8154-082f737b389b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:52.296927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeeb26-e0bb-472b-8154-082f737b389b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:52.301643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeeb26-e0bb-472b-8154-082f737b389b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:52.362123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-54eeeb26-e0bb-472b-8154-082f737b389b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:56:52.363440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-54eeeb26-e0bb-472b-8154-082f737b389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:52.458805 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 656/1319] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:56:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:56:52.747824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6e615-1a9b-4742-8ec6-18b1030194fe None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:56:52.749383 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e4f6e615-1a9b-4742-8ec6-18b1030194fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:52.851008 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6e615-1a9b-4742-8ec6-18b1030194fe None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:56:52.922294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85a0b7-3172-47b0-9330-377680ee41de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:56:52.922913 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85a0b7-3172-47b0-9330-377680ee41de service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:52.923188 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85a0b7-3172-47b0-9330-377680ee41de service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:52.923452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85a0b7-3172-47b0-9330-377680ee41de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:52.925306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4a85a0b7-3172-47b0-9330-377680ee41de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:52.932004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:53.031978 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85a0b7-3172-47b0-9330-377680ee41de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:56:53.033355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85a0b7-3172-47b0-9330-377680ee41de service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:56:53.051087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 664/1320] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:56:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:56:56.137690 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001abe69-c75a-42c4-8ae5-330411af8e7c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:56.138196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001abe69-c75a-42c4-8ae5-330411af8e7c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:56.138196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001abe69-c75a-42c4-8ae5-330411af8e7c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:56.139503 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 657/1321] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:56:56.144617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce8950-7004-47e3-9475-f6c5ff26456b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:56.145058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce8950-7004-47e3-9475-f6c5ff26456b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:56.145167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce8950-7004-47e3-9475-f6c5ff26456b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:56.210079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4fce8950-7004-47e3-9475-f6c5ff26456b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:56:56.211302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4fce8950-7004-47e3-9475-f6c5ff26456b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:56.268737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:56.307365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 665/1322] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:56:56.591081 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001abe69-c75a-42c4-8ae5-330411af8e7c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:56:56.592690 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-001abe69-c75a-42c4-8ae5-330411af8e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:56.612744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:56.695578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001abe69-c75a-42c4-8ae5-330411af8e7c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:56:56.771330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e6b80-69c4-4428-999f-7da566fca2a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:56:56.771942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6b80-69c4-4428-999f-7da566fca2a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:56.772214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6b80-69c4-4428-999f-7da566fca2a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:56.772479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6b80-69c4-4428-999f-7da566fca2a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:56.774070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-192e6b80-69c4-4428-999f-7da566fca2a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:56.846662 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:56.884418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e6b80-69c4-4428-999f-7da566fca2a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:56:56.885863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e6b80-69c4-4428-999f-7da566fca2a4 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:56:56.904323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 658/1323] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:56:59.082519 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c55ee-c279-4a82-84e4-2881221c029c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:59.083102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c55ee-c279-4a82-84e4-2881221c029c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:59.083102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c55ee-c279-4a82-84e4-2881221c029c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:59.084440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 666/1324] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:56:59.089822 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f288a9-1a0d-482b-b4b7-06166cd45c3d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:59.090240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f288a9-1a0d-482b-b4b7-06166cd45c3d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:59.090379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f288a9-1a0d-482b-b4b7-06166cd45c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:59.156851 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-08f288a9-1a0d-482b-b4b7-06166cd45c3d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:56:59.158072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-08f288a9-1a0d-482b-b4b7-06166cd45c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:59.259969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 659/1325] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:56:59.564971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab0f74-5513-4887-8ee1-b1626d4cc2a5 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:59.565328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab0f74-5513-4887-8ee1-b1626d4cc2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:59.565614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab0f74-5513-4887-8ee1-b1626d4cc2a5 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:59.567048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 667/1326] 10.222.0.59 () {54 vars in 965 bytes} [Fri Sep 16 07:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:56:59.573092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665ed99-c660-4842-a759-c65a2583f5d7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:59.573372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665ed99-c660-4842-a759-c65a2583f5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:59.573642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665ed99-c660-4842-a759-c65a2583f5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:59.605720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:56:59.650852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0665ed99-c660-4842-a759-c65a2583f5d7 None None] MFA Rules not processed for user `a29c4e53ac9a438e9701a70f3de3f32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:56:59.652282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0665ed99-c660-4842-a759-c65a2583f5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:59.730348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 660/1327] 10.222.0.59 () {58 vars in 1062 bytes} [Fri Sep 16 07:56:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 07:56:59.740251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab0f74-5513-4887-8ee1-b1626d4cc2a5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:56:59.741501 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-02ab0f74-5513-4887-8ee1-b1626d4cc2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:59.844730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab0f74-5513-4887-8ee1-b1626d4cc2a5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:56:59.917283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cbce5-497a-4c7a-9043-e844e806d303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:56:59.917812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cbce5-497a-4c7a-9043-e844e806d303 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:56:59.918091 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cbce5-497a-4c7a-9043-e844e806d303 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:56:59.918360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cbce5-497a-4c7a-9043-e844e806d303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:56:59.919991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-407cbce5-497a-4c7a-9043-e844e806d303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:56:59.936883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:00.029911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cbce5-497a-4c7a-9043-e844e806d303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:00.031375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cbce5-497a-4c7a-9043-e844e806d303 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:00.049032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 668/1328] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:02.478614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a646bc-4995-436d-86f2-aaaa6b897aa4 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:02.479165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a646bc-4995-436d-86f2-aaaa6b897aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:02.479282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a646bc-4995-436d-86f2-aaaa6b897aa4 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:02.480587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 661/1329] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:02.485814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847d794-6c18-4800-b9b8-c1bcbb05dcf5 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:02.486206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847d794-6c18-4800-b9b8-c1bcbb05dcf5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:02.486370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847d794-6c18-4800-b9b8-c1bcbb05dcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:02.551751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c847d794-6c18-4800-b9b8-c1bcbb05dcf5 None None] MFA Rules not processed for user `71f957f6c8724676af13c99ddc4b62e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:02.552997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c847d794-6c18-4800-b9b8-c1bcbb05dcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:02.622768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:02.656315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 669/1330] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:02.937422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a646bc-4995-436d-86f2-aaaa6b897aa4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:02.938992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d6a646bc-4995-436d-86f2-aaaa6b897aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:03.041735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a646bc-4995-436d-86f2-aaaa6b897aa4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:03.062949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:03.115841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8262d-753e-4d4c-aefc-323664b765a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:03.116377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8262d-753e-4d4c-aefc-323664b765a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:03.116652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8262d-753e-4d4c-aefc-323664b765a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:03.116925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8262d-753e-4d4c-aefc-323664b765a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:03.119088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-31a8262d-753e-4d4c-aefc-323664b765a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:03.226938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8262d-753e-4d4c-aefc-323664b765a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:03.228303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8262d-753e-4d4c-aefc-323664b765a5 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:03.246398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 662/1331] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:05.844846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e7146-38f9-4a43-bd7f-4c23dc06a978 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:05.845370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e7146-38f9-4a43-bd7f-4c23dc06a978 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:05.845484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e7146-38f9-4a43-bd7f-4c23dc06a978 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:05.846782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 670/1332] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:05.852647 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7c950-5e99-4596-a482-ac64d521b284 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:05.853052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7c950-5e99-4596-a482-ac64d521b284 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:05.853212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7c950-5e99-4596-a482-ac64d521b284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:05.915695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:05.918227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-77a7c950-5e99-4596-a482-ac64d521b284 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:05.919453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-77a7c950-5e99-4596-a482-ac64d521b284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:06.015613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 663/1333] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:06.302371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e7146-38f9-4a43-bd7f-4c23dc06a978 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:06.303952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8d3e7146-38f9-4a43-bd7f-4c23dc06a978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:06.405744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e7146-38f9-4a43-bd7f-4c23dc06a978 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:06.479136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef1b4e-b3a7-4bf1-8999-8b4dea842e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:06.479738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef1b4e-b3a7-4bf1-8999-8b4dea842e87 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:06.480015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef1b4e-b3a7-4bf1-8999-8b4dea842e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:06.480288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef1b4e-b3a7-4bf1-8999-8b4dea842e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:06.481876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-20ef1b4e-b3a7-4bf1-8999-8b4dea842e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:06.507952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:06.589379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef1b4e-b3a7-4bf1-8999-8b4dea842e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:06.590760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef1b4e-b3a7-4bf1-8999-8b4dea842e87 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:06.608673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 671/1334] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:08.802738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752ea62-f73c-4b70-8d2a-6e73c765a43e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:08.803240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752ea62-f73c-4b70-8d2a-6e73c765a43e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:08.803361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752ea62-f73c-4b70-8d2a-6e73c765a43e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:08.804632 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 664/1335] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:08.809814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db58a27-f3d3-408a-8e38-990c103855fe None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:08.810239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db58a27-f3d3-408a-8e38-990c103855fe None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:08.810373 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db58a27-f3d3-408a-8e38-990c103855fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:08.875194 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6db58a27-f3d3-408a-8e38-990c103855fe None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:08.876408 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6db58a27-f3d3-408a-8e38-990c103855fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:08.950082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:08.972015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 672/1336] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:09.258036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752ea62-f73c-4b70-8d2a-6e73c765a43e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:09.260151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7752ea62-f73c-4b70-8d2a-6e73c765a43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:09.361046 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752ea62-f73c-4b70-8d2a-6e73c765a43e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:09.389675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:09.433745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1504621-8ffb-49a9-951d-7a7a7c562c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:09.434348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1504621-8ffb-49a9-951d-7a7a7c562c5b service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:09.434624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1504621-8ffb-49a9-951d-7a7a7c562c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:09.434920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1504621-8ffb-49a9-951d-7a7a7c562c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:09.436515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a1504621-8ffb-49a9-951d-7a7a7c562c5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:09.544823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1504621-8ffb-49a9-951d-7a7a7c562c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:09.546188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1504621-8ffb-49a9-951d-7a7a7c562c5b service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:09.564714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 665/1337] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:12.244455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ddacb-59dd-4db0-921d-cefed73c6136 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:12.245023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ddacb-59dd-4db0-921d-cefed73c6136 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:12.245023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ddacb-59dd-4db0-921d-cefed73c6136 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:12.246357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 673/1338] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:12.251677 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653093d4-8188-40e6-9f4c-8e2b77dd2a03 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:12.252098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653093d4-8188-40e6-9f4c-8e2b77dd2a03 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:12.252233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653093d4-8188-40e6-9f4c-8e2b77dd2a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:12.317848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-653093d4-8188-40e6-9f4c-8e2b77dd2a03 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:12.319000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-653093d4-8188-40e6-9f4c-8e2b77dd2a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:12.327226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:12.414092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 666/1339] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:12.694081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ddacb-59dd-4db0-921d-cefed73c6136 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:12.695647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c35ddacb-59dd-4db0-921d-cefed73c6136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:12.797287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ddacb-59dd-4db0-921d-cefed73c6136 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:12.869759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319379e4-ada7-4fc0-b9d9-62290339f506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:12.870360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319379e4-ada7-4fc0-b9d9-62290339f506 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:12.870640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319379e4-ada7-4fc0-b9d9-62290339f506 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:12.870937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319379e4-ada7-4fc0-b9d9-62290339f506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:12.872544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-319379e4-ada7-4fc0-b9d9-62290339f506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:12.907121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:12.979044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319379e4-ada7-4fc0-b9d9-62290339f506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:12.980402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319379e4-ada7-4fc0-b9d9-62290339f506 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:12.998778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 674/1340] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:17.208234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cfacd-2184-4ce5-bdb7-526164b76abb None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:17.208838 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cfacd-2184-4ce5-bdb7-526164b76abb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:17.208838 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cfacd-2184-4ce5-bdb7-526164b76abb None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:17.210118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 667/1341] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:17.216013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb906a98-801a-4f00-a54c-205d66fbbc19 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:17.216456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb906a98-801a-4f00-a54c-205d66fbbc19 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:17.216588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb906a98-801a-4f00-a54c-205d66fbbc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:17.290071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-eb906a98-801a-4f00-a54c-205d66fbbc19 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:17.291459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-eb906a98-801a-4f00-a54c-205d66fbbc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:17.386435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:17.406849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 675/1342] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:17.682938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cfacd-2184-4ce5-bdb7-526164b76abb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:17.684502 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1d1cfacd-2184-4ce5-bdb7-526164b76abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:17.793881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cfacd-2184-4ce5-bdb7-526164b76abb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:17.830939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:17.866614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05800f6c-7414-4911-903d-d4d486832105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:17.867256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05800f6c-7414-4911-903d-d4d486832105 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:17.867536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05800f6c-7414-4911-903d-d4d486832105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:17.867804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05800f6c-7414-4911-903d-d4d486832105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:17.869455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-05800f6c-7414-4911-903d-d4d486832105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:17.978771 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05800f6c-7414-4911-903d-d4d486832105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:17.980175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05800f6c-7414-4911-903d-d4d486832105 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:17.998131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 668/1343] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:21.144018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f0040-3f38-40c0-986b-3fbbba057e02 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:21.144619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f0040-3f38-40c0-986b-3fbbba057e02 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:21.144619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f0040-3f38-40c0-986b-3fbbba057e02 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:21.145947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 676/1344] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:21.152003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de923-2027-4c86-8b4f-d38734799198 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:21.152582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de923-2027-4c86-8b4f-d38734799198 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:21.152582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de923-2027-4c86-8b4f-d38734799198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:21.223567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-258de923-2027-4c86-8b4f-d38734799198 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:21.224777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-258de923-2027-4c86-8b4f-d38734799198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:21.244018 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:21.325322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 669/1345] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:21.615911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f0040-3f38-40c0-986b-3fbbba057e02 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:21.617653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-185f0040-3f38-40c0-986b-3fbbba057e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:21.727071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f0040-3f38-40c0-986b-3fbbba057e02 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:21.805026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f8b81-920d-481f-94fe-3cd56c9088e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:21.805629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f8b81-920d-481f-94fe-3cd56c9088e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:21.805905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f8b81-920d-481f-94fe-3cd56c9088e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:21.806790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f8b81-920d-481f-94fe-3cd56c9088e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:21.808434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0f4f8b81-920d-481f-94fe-3cd56c9088e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:21.874360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:21.948797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f8b81-920d-481f-94fe-3cd56c9088e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:21.950351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f8b81-920d-481f-94fe-3cd56c9088e9 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:21.970054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 677/1346] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:26.276384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024267cf-0a52-425a-949a-e4605335eb26 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:26.276909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024267cf-0a52-425a-949a-e4605335eb26 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:26.276909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024267cf-0a52-425a-949a-e4605335eb26 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:26.278237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 670/1347] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:26.283476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80449344-2209-4d3a-8774-89bf8c93f872 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:26.283890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80449344-2209-4d3a-8774-89bf8c93f872 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:26.284009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80449344-2209-4d3a-8774-89bf8c93f872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:26.351558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-80449344-2209-4d3a-8774-89bf8c93f872 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:26.357003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-80449344-2209-4d3a-8774-89bf8c93f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:26.452251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:26.454452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 678/1348] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:26.739387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024267cf-0a52-425a-949a-e4605335eb26 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:26.740944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-024267cf-0a52-425a-949a-e4605335eb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:26.843490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024267cf-0a52-425a-949a-e4605335eb26 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:26.890658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:26.915173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848ac99-2c7b-4c96-9754-a920dd214913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:26.915782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848ac99-2c7b-4c96-9754-a920dd214913 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:26.916296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848ac99-2c7b-4c96-9754-a920dd214913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:26.916751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848ac99-2c7b-4c96-9754-a920dd214913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:26.918608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4848ac99-2c7b-4c96-9754-a920dd214913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:27.023239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848ac99-2c7b-4c96-9754-a920dd214913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:27.025171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848ac99-2c7b-4c96-9754-a920dd214913 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:27.042890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 671/1349] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:32.032283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec606b87-dbd6-4cae-a826-3b1a0f70b5f7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:32.033522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec606b87-dbd6-4cae-a826-3b1a0f70b5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:32.033802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec606b87-dbd6-4cae-a826-3b1a0f70b5f7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:32.035243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 679/1350] 10.222.0.59 () {54 vars in 921 bytes} [Fri Sep 16 07:57:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:32.040964 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d241736-5cc7-4ad6-9812-add0e968db09 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:32.041403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d241736-5cc7-4ad6-9812-add0e968db09 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:32.041511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d241736-5cc7-4ad6-9812-add0e968db09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:32.141002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3d241736-5cc7-4ad6-9812-add0e968db09 None None] MFA Rules not processed for user `c2dca05dfe8b4af0a02ea93fbe29c556`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:32.142293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3d241736-5cc7-4ad6-9812-add0e968db09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:32.184669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:32.335995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 672/1351] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Sep 16 07:57:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:36.514474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085bf68-e971-470a-94e0-6917dd951258 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:36.515464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085bf68-e971-470a-94e0-6917dd951258 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:36.516101 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085bf68-e971-470a-94e0-6917dd951258 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:36.517530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 680/1352] 10.222.0.59 () {54 vars in 921 bytes} [Fri Sep 16 07:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:36.522849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503af4fb-8a16-40fa-b05e-d03a36ab9542 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:36.523267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503af4fb-8a16-40fa-b05e-d03a36ab9542 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:36.523405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503af4fb-8a16-40fa-b05e-d03a36ab9542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:36.605137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-503af4fb-8a16-40fa-b05e-d03a36ab9542 None None] MFA Rules not processed for user `c2dca05dfe8b4af0a02ea93fbe29c556`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:36.606356 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-503af4fb-8a16-40fa-b05e-d03a36ab9542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:36.684566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:36.734720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 673/1353] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Sep 16 07:57:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:37.964136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf8a7d-df3e-485f-b517-bafecc5c06fd None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:37.964675 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf8a7d-df3e-485f-b517-bafecc5c06fd None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:37.964675 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf8a7d-df3e-485f-b517-bafecc5c06fd None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:37.966275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 681/1354] 10.222.0.59 () {54 vars in 921 bytes} [Fri Sep 16 07:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:37.971298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e586dd8-bf74-479b-9b42-a4b478e565ac None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:37.971756 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e586dd8-bf74-479b-9b42-a4b478e565ac None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:37.971997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e586dd8-bf74-479b-9b42-a4b478e565ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:38.049083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4e586dd8-bf74-479b-9b42-a4b478e565ac None None] MFA Rules not processed for user `c2dca05dfe8b4af0a02ea93fbe29c556`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:38.050200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4e586dd8-bf74-479b-9b42-a4b478e565ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:38.156011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 674/1355] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Sep 16 07:57:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:41.694173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4944daa-58a2-43be-93fe-81d3de578382 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:41.695184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4944daa-58a2-43be-93fe-81d3de578382 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:41.695652 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4944daa-58a2-43be-93fe-81d3de578382 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:41.698028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 682/1356] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:41.703927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c75dcc-3689-422f-9699-818a203f729d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:41.704410 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c75dcc-3689-422f-9699-818a203f729d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:41.704528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c75dcc-3689-422f-9699-818a203f729d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:41.749511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:41.787266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b3c75dcc-3689-422f-9699-818a203f729d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:41.788403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b3c75dcc-3689-422f-9699-818a203f729d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:41.894070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 675/1357] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:42.196630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724153e-e0a0-4e38-8c6d-a96c9f9a2dac None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:42.196911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724153e-e0a0-4e38-8c6d-a96c9f9a2dac None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:42.197180 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724153e-e0a0-4e38-8c6d-a96c9f9a2dac None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:42.198883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 683/1358] 10.222.0.59 () {54 vars in 962 bytes} [Fri Sep 16 07:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:42.208047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62a357-8715-4b93-a5e6-96b235f2cc88 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:42.208344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62a357-8715-4b93-a5e6-96b235f2cc88 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:42.208618 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62a357-8715-4b93-a5e6-96b235f2cc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:42.294820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4a62a357-8715-4b93-a5e6-96b235f2cc88 None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:42.296216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4a62a357-8715-4b93-a5e6-96b235f2cc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:42.352295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:42.425345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 676/1359] 10.222.0.59 () {58 vars in 1059 bytes} [Fri Sep 16 07:57:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:42.433596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724153e-e0a0-4e38-8c6d-a96c9f9a2dac None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:42.436134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a724153e-e0a0-4e38-8c6d-a96c9f9a2dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:42.600307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724153e-e0a0-4e38-8c6d-a96c9f9a2dac None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:42.679277 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:42.722141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f65433-bfd3-4b4b-924f-c7910b2f99f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:42.723102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f65433-bfd3-4b4b-924f-c7910b2f99f8 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:42.723348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f65433-bfd3-4b4b-924f-c7910b2f99f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:42.723625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f65433-bfd3-4b4b-924f-c7910b2f99f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:42.725275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d3f65433-bfd3-4b4b-924f-c7910b2f99f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:42.837938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f65433-bfd3-4b4b-924f-c7910b2f99f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:42.839702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f65433-bfd3-4b4b-924f-c7910b2f99f8 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:42.859561 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 684/1360] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 07:57:45.265158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add72ad-b4a3-4b8f-9fac-a678d0f45057 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:45.265158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add72ad-b4a3-4b8f-9fac-a678d0f45057 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:45.265158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add72ad-b4a3-4b8f-9fac-a678d0f45057 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:45.265158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 677/1361] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:45.266115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efbc16-ecc4-41b1-bba1-dddc57b65439 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:45.266115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efbc16-ecc4-41b1-bba1-dddc57b65439 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:45.266115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efbc16-ecc4-41b1-bba1-dddc57b65439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:45.329673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-61efbc16-ecc4-41b1-bba1-dddc57b65439 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:45.330937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-61efbc16-ecc4-41b1-bba1-dddc57b65439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:45.380968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:45.446930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 685/1362] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:45.735078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add72ad-b4a3-4b8f-9fac-a678d0f45057 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:45.741498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7add72ad-b4a3-4b8f-9fac-a678d0f45057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:45.914123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add72ad-b4a3-4b8f-9fac-a678d0f45057 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:45.927104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:46.064778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001069ac-3a21-4a8a-abf3-fc56925bc7a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:46.065385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001069ac-3a21-4a8a-abf3-fc56925bc7a3 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:46.065668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001069ac-3a21-4a8a-abf3-fc56925bc7a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:46.065934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001069ac-3a21-4a8a-abf3-fc56925bc7a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:46.067546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-001069ac-3a21-4a8a-abf3-fc56925bc7a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:46.161905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001069ac-3a21-4a8a-abf3-fc56925bc7a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:46.163490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001069ac-3a21-4a8a-abf3-fc56925bc7a3 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:46.182554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 678/1363] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:48.770590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f2e5c-dbe0-46b3-accb-f5d702a8f62c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:48.771485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f2e5c-dbe0-46b3-accb-f5d702a8f62c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:48.771777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f2e5c-dbe0-46b3-accb-f5d702a8f62c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:48.773194 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 686/1364] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:48.779314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421daf-2d40-4a1e-a168-459d7b1d2294 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:48.779738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421daf-2d40-4a1e-a168-459d7b1d2294 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:48.780058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c421daf-2d40-4a1e-a168-459d7b1d2294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:48.796187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:48.852324 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1c421daf-2d40-4a1e-a168-459d7b1d2294 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:48.853687 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1c421daf-2d40-4a1e-a168-459d7b1d2294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:48.959575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 679/1365] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:49.248603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa754c3-2811-415b-8a89-7ee7e9e70e2a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:49.249412 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa754c3-2811-415b-8a89-7ee7e9e70e2a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:49.249792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa754c3-2811-415b-8a89-7ee7e9e70e2a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:49.251437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 687/1366] 10.222.0.59 () {54 vars in 962 bytes} [Fri Sep 16 07:57:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:49.257188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541444c1-2e4d-409e-976f-14c8f7e67d9a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:49.261620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541444c1-2e4d-409e-976f-14c8f7e67d9a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:49.261620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541444c1-2e4d-409e-976f-14c8f7e67d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:49.339977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-541444c1-2e4d-409e-976f-14c8f7e67d9a None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:49.341188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-541444c1-2e4d-409e-976f-14c8f7e67d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:49.377017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:49.465461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 680/1367] 10.222.0.59 () {58 vars in 1059 bytes} [Fri Sep 16 07:57:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:49.473674 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa754c3-2811-415b-8a89-7ee7e9e70e2a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:49.475315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6fa754c3-2811-415b-8a89-7ee7e9e70e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:49.629261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa754c3-2811-415b-8a89-7ee7e9e70e2a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:49.652835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:49.760430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf5299-b52d-46bc-b748-6f8bfce1e2a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:49.761038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf5299-b52d-46bc-b748-6f8bfce1e2a5 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:49.761313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf5299-b52d-46bc-b748-6f8bfce1e2a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:49.761583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf5299-b52d-46bc-b748-6f8bfce1e2a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:49.763281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c2bf5299-b52d-46bc-b748-6f8bfce1e2a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:49.862649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf5299-b52d-46bc-b748-6f8bfce1e2a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:49.864300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf5299-b52d-46bc-b748-6f8bfce1e2a5 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:49.884268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 688/1368] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:49.896697 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b8fd1-3e74-49bc-a4b2-4c15cc57222b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:49.897146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b8fd1-3e74-49bc-a4b2-4c15cc57222b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:49.897249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b8fd1-3e74-49bc-a4b2-4c15cc57222b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:49.898722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 681/1369] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:49.903873 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f4a2d-035d-4e40-837d-f6b8ae299515 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:49.904157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f4a2d-035d-4e40-837d-f6b8ae299515 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:49.904433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f4a2d-035d-4e40-837d-f6b8ae299515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:49.937252 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:49.971116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-da9f4a2d-035d-4e40-837d-f6b8ae299515 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:49.972365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-da9f4a2d-035d-4e40-837d-f6b8ae299515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:50.080743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 689/1370] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:50.384361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa002403-83c8-40c6-b5aa-52086ac36744 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:50.384751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa002403-83c8-40c6-b5aa-52086ac36744 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:50.385184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa002403-83c8-40c6-b5aa-52086ac36744 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:50.386789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 682/1371] 10.222.0.59 () {54 vars in 964 bytes} [Fri Sep 16 07:57:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:50.391914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32e689-79bc-4d58-bae5-e36a21c465e4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:50.392310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32e689-79bc-4d58-bae5-e36a21c465e4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:50.392705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32e689-79bc-4d58-bae5-e36a21c465e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:50.474461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0b32e689-79bc-4d58-bae5-e36a21c465e4 None None] MFA Rules not processed for user `fb514134abe54e5ca8125e572a2ff928`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:50.475839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0b32e689-79bc-4d58-bae5-e36a21c465e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:50.525316 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:50.558261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 690/1372] 10.222.0.59 () {58 vars in 1061 bytes} [Fri Sep 16 07:57:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:50.566061 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa002403-83c8-40c6-b5aa-52086ac36744 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:50.567714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-aa002403-83c8-40c6-b5aa-52086ac36744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:50.683769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa002403-83c8-40c6-b5aa-52086ac36744 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:50.730438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:50.769707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04923792-c521-4131-a097-81c8aaa40afb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:50.770406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04923792-c521-4131-a097-81c8aaa40afb service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:50.770711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04923792-c521-4131-a097-81c8aaa40afb service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:50.770987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04923792-c521-4131-a097-81c8aaa40afb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:50.772593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-04923792-c521-4131-a097-81c8aaa40afb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:50.900848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04923792-c521-4131-a097-81c8aaa40afb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:50.902427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04923792-c521-4131-a097-81c8aaa40afb service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:50.921191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 683/1373] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:52.254398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb733b-8ac3-4b42-8cad-86af1cfb9b4a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:52.255007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb733b-8ac3-4b42-8cad-86af1cfb9b4a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:52.255007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb733b-8ac3-4b42-8cad-86af1cfb9b4a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:52.256345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 691/1374] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:52.262176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6e4a1-1ae3-46e8-a67a-61b168b75976 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:52.262610 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6e4a1-1ae3-46e8-a67a-61b168b75976 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:52.262823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6e4a1-1ae3-46e8-a67a-61b168b75976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:52.329868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c1d6e4a1-1ae3-46e8-a67a-61b168b75976 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:52.331123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c1d6e4a1-1ae3-46e8-a67a-61b168b75976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:52.352097 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:52.427316 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 684/1375] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:52.718381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb733b-8ac3-4b42-8cad-86af1cfb9b4a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:52.720044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-59eb733b-8ac3-4b42-8cad-86af1cfb9b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:52.874529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb733b-8ac3-4b42-8cad-86af1cfb9b4a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:52.928773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:52.991432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29529c8b-23cf-4562-924a-fdd02ea32d6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:52.992084 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29529c8b-23cf-4562-924a-fdd02ea32d6d service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:52.992990 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29529c8b-23cf-4562-924a-fdd02ea32d6d service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:52.993266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29529c8b-23cf-4562-924a-fdd02ea32d6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:52.995027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-29529c8b-23cf-4562-924a-fdd02ea32d6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:53.088723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29529c8b-23cf-4562-924a-fdd02ea32d6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:53.090372 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29529c8b-23cf-4562-924a-fdd02ea32d6d service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:53.111240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 692/1376] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:55.421913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08340900-3343-4eea-ae3d-6a0ccae86923 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:55.422509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08340900-3343-4eea-ae3d-6a0ccae86923 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:55.422509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08340900-3343-4eea-ae3d-6a0ccae86923 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:55.424337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 685/1377] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:55.430108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790e807-4cdc-4c43-9c08-cc13524bd526 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:55.430649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790e807-4cdc-4c43-9c08-cc13524bd526 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:55.430649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790e807-4cdc-4c43-9c08-cc13524bd526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:55.499193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8790e807-4cdc-4c43-9c08-cc13524bd526 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:55.500417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8790e807-4cdc-4c43-9c08-cc13524bd526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:55.508276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:55.607509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 693/1378] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:55.894416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08340900-3343-4eea-ae3d-6a0ccae86923 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:55.896186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-08340900-3343-4eea-ae3d-6a0ccae86923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:56.049119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08340900-3343-4eea-ae3d-6a0ccae86923 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:56.053220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:56.167132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280315bc-4c53-47a6-b5cf-26ee50c3405c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:56.167821 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280315bc-4c53-47a6-b5cf-26ee50c3405c service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:56.168198 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280315bc-4c53-47a6-b5cf-26ee50c3405c service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:56.168567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280315bc-4c53-47a6-b5cf-26ee50c3405c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:56.170310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-280315bc-4c53-47a6-b5cf-26ee50c3405c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:56.267271 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280315bc-4c53-47a6-b5cf-26ee50c3405c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:56.269038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280315bc-4c53-47a6-b5cf-26ee50c3405c service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:56.288851 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 686/1379] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:57:58.731503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8f825-c305-4167-8309-26a1bf656f66 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:58.732074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8f825-c305-4167-8309-26a1bf656f66 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:58.732074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8f825-c305-4167-8309-26a1bf656f66 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:58.733396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 694/1380] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:57:58.739012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4a22-5c61-4713-9aee-12a91e0da1ae None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:58.739473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4a22-5c61-4713-9aee-12a91e0da1ae None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:58.739648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4a22-5c61-4713-9aee-12a91e0da1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:58.753066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:58.811504 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7a8e4a22-5c61-4713-9aee-12a91e0da1ae None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:57:58.812703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7a8e4a22-5c61-4713-9aee-12a91e0da1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:58.911984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 687/1381] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:57:59.200599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8f825-c305-4167-8309-26a1bf656f66 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:57:59.202148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-90b8f825-c305-4167-8309-26a1bf656f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:59.339653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:59.353006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8f825-c305-4167-8309-26a1bf656f66 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:57:59.484846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0baf44-7354-4038-b279-05a9497b4f65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:57:59.485380 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0baf44-7354-4038-b279-05a9497b4f65 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:57:59.485662 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0baf44-7354-4038-b279-05a9497b4f65 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:57:59.485935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0baf44-7354-4038-b279-05a9497b4f65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:57:59.487583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7c0baf44-7354-4038-b279-05a9497b4f65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:57:59.588916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0baf44-7354-4038-b279-05a9497b4f65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:57:59.591102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0baf44-7354-4038-b279-05a9497b4f65 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:57:59.604505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:57:59.612811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 695/1382] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:02.310565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c013b-1945-4279-811c-3c9840be6594 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:02.322734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c013b-1945-4279-811c-3c9840be6594 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:02.322734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c013b-1945-4279-811c-3c9840be6594 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:02.322734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 688/1383] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:02.323009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406b91c-c300-4857-b283-62b236a9f204 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:02.323009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406b91c-c300-4857-b283-62b236a9f204 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:02.323009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406b91c-c300-4857-b283-62b236a9f204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:02.391689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2406b91c-c300-4857-b283-62b236a9f204 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:02.392968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2406b91c-c300-4857-b283-62b236a9f204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:02.493976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 696/1384] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:02.784772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c013b-1945-4279-811c-3c9840be6594 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:02.786302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6c3c013b-1945-4279-811c-3c9840be6594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:02.874357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:02.937030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c013b-1945-4279-811c-3c9840be6594 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:03.058183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d69e62-bb98-4504-83e5-151ed7be5fd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:03.058830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d69e62-bb98-4504-83e5-151ed7be5fd1 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:03.059112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d69e62-bb98-4504-83e5-151ed7be5fd1 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:03.059384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d69e62-bb98-4504-83e5-151ed7be5fd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:03.061026 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f4d69e62-bb98-4504-83e5-151ed7be5fd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:03.117005 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:03.155602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d69e62-bb98-4504-83e5-151ed7be5fd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:03.157166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d69e62-bb98-4504-83e5-151ed7be5fd1 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:03.177067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 689/1385] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:05.333110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babceca-5bc1-4ccc-abfd-66fb55f57d8c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:05.333614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babceca-5bc1-4ccc-abfd-66fb55f57d8c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:05.333711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babceca-5bc1-4ccc-abfd-66fb55f57d8c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:05.335043 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 697/1386] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:05.340352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6eb85e-aeae-4bb4-8073-37440c3a4e56 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:05.340748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6eb85e-aeae-4bb4-8073-37440c3a4e56 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:05.340907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6eb85e-aeae-4bb4-8073-37440c3a4e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:05.407996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-cd6eb85e-aeae-4bb4-8073-37440c3a4e56 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:05.409200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cd6eb85e-aeae-4bb4-8073-37440c3a4e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:05.505363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 690/1387] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:05.797811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babceca-5bc1-4ccc-abfd-66fb55f57d8c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:05.799720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8babceca-5bc1-4ccc-abfd-66fb55f57d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:05.883228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:05.959691 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babceca-5bc1-4ccc-abfd-66fb55f57d8c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:06.074178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d658d-325b-41dc-88f0-6bba827fcc5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:06.074719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d658d-325b-41dc-88f0-6bba827fcc5e service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:06.074998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d658d-325b-41dc-88f0-6bba827fcc5e service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:06.075264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d658d-325b-41dc-88f0-6bba827fcc5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:06.076853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e10d658d-325b-41dc-88f0-6bba827fcc5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:06.115859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:06.170245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d658d-325b-41dc-88f0-6bba827fcc5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:06.171874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d658d-325b-41dc-88f0-6bba827fcc5e service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:06.191527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 698/1388] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:06.453201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1869e1c-cf24-4f9b-aaa4-50f56a1a6f3e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:06.453791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1869e1c-cf24-4f9b-aaa4-50f56a1a6f3e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:06.453791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1869e1c-cf24-4f9b-aaa4-50f56a1a6f3e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:06.455144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 691/1389] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:06.461093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2f887-24e3-4e3c-a69b-a2ed825a0dc5 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:06.461645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2f887-24e3-4e3c-a69b-a2ed825a0dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:06.461645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2f887-24e3-4e3c-a69b-a2ed825a0dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:06.536641 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a0b2f887-24e3-4e3c-a69b-a2ed825a0dc5 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:06.537877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a0b2f887-24e3-4e3c-a69b-a2ed825a0dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:06.637478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 699/1390] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:06.938974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987b4be-21c4-41c6-b188-40b0c6e76506 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:06.939325 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987b4be-21c4-41c6-b188-40b0c6e76506 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:06.939666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987b4be-21c4-41c6-b188-40b0c6e76506 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:06.941050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 692/1391] 10.222.0.59 () {54 vars in 962 bytes} [Fri Sep 16 07:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:06.946980 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40ae09-a7db-45c0-81c2-031368178d79 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:06.947268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40ae09-a7db-45c0-81c2-031368178d79 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:06.947544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40ae09-a7db-45c0-81c2-031368178d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:06.963431 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:07.025987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9b40ae09-a7db-45c0-81c2-031368178d79 None None] MFA Rules not processed for user `c214b2c97a7b41698b433eac8379442a`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:07.027250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9b40ae09-a7db-45c0-81c2-031368178d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:07.103856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 700/1392] 10.222.0.59 () {58 vars in 1059 bytes} [Fri Sep 16 07:58:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:07.112878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987b4be-21c4-41c6-b188-40b0c6e76506 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:07.114400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d987b4be-21c4-41c6-b188-40b0c6e76506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:07.146579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:07.222593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987b4be-21c4-41c6-b188-40b0c6e76506 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:07.296906 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281ee0f-bc10-471f-97c5-80c30b9cc851 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c214b2c97a7b41698b433eac8379442a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:07.297519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281ee0f-bc10-471f-97c5-80c30b9cc851 service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:07.297802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281ee0f-bc10-471f-97c5-80c30b9cc851 service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:07.298332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281ee0f-bc10-471f-97c5-80c30b9cc851 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:07.300089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e281ee0f-bc10-471f-97c5-80c30b9cc851 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:07.380231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:07.412765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281ee0f-bc10-471f-97c5-80c30b9cc851 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:07.415255 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281ee0f-bc10-471f-97c5-80c30b9cc851 service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:07.435555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 693/1393] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:08.319101 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5d006-7f5f-4c61-9a79-3bf16b32813d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:08.319610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5d006-7f5f-4c61-9a79-3bf16b32813d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:08.319849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5d006-7f5f-4c61-9a79-3bf16b32813d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:08.392863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cba5d006-7f5f-4c61-9a79-3bf16b32813d None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:08.394284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cba5d006-7f5f-4c61-9a79-3bf16b32813d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:08.452190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:08.483593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 701/1394] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 07:58:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:08.494591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5c8ff-4668-4ee7-bfce-d41a877615c9 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:08.495006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5c8ff-4668-4ee7-bfce-d41a877615c9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:08.495203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5c8ff-4668-4ee7-bfce-d41a877615c9 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:08.496610 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 694/1395] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:08.502103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb173e6-1ec2-4ee0-b04e-4c0f7a91f902 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:08.502394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb173e6-1ec2-4ee0-b04e-4c0f7a91f902 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:08.502672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb173e6-1ec2-4ee0-b04e-4c0f7a91f902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:08.527324 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dc45a-adaf-40fe-a3ee-49d0d33e0e53 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:08.527607 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dc45a-adaf-40fe-a3ee-49d0d33e0e53 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:08.527879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dc45a-adaf-40fe-a3ee-49d0d33e0e53 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:08.529207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 695/1396] 10.222.0.59 () {54 vars in 943 bytes} [Fri Sep 16 07:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:08.535304 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec6040-0013-4a2d-85bf-cb75bdab2e35 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:08.535590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec6040-0013-4a2d-85bf-cb75bdab2e35 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:08.535869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec6040-0013-4a2d-85bf-cb75bdab2e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:08.573445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6fb173e6-1ec2-4ee0-b04e-4c0f7a91f902 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:08.574806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6fb173e6-1ec2-4ee0-b04e-4c0f7a91f902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:08.613345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0cec6040-0013-4a2d-85bf-cb75bdab2e35 None None] MFA Rules not processed for user `a1ae39fdd6ff4841843c9826dd88229a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:08.615012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0cec6040-0013-4a2d-85bf-cb75bdab2e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:08.674169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 702/1397] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:08.724794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 696/1398] 10.222.0.59 () {58 vars in 1040 bytes} [Fri Sep 16 07:58:08 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 07:58:08.734301 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb173e6-1ec2-4ee0-b04e-4c0f7a91f902 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:08.735613 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6fb173e6-1ec2-4ee0-b04e-4c0f7a91f902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:08.788569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:08.839560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb173e6-1ec2-4ee0-b04e-4c0f7a91f902 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:08.940383 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a9274-c633-4c32-b946-2a49fe895c54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:08.941014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a9274-c633-4c32-b946-2a49fe895c54 service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:08.941289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a9274-c633-4c32-b946-2a49fe895c54 service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:08.941557 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a9274-c633-4c32-b946-2a49fe895c54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:08.943272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a75a9274-c633-4c32-b946-2a49fe895c54 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:08.962391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec6040-0013-4a2d-85bf-cb75bdab2e35 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:08.963731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0cec6040-0013-4a2d-85bf-cb75bdab2e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:08.981024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:09.014940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:09.033085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a9274-c633-4c32-b946-2a49fe895c54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:09.034844 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a9274-c633-4c32-b946-2a49fe895c54 service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:09.043456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 703/1399] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 07:58:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 07:58:09.062404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec6040-0013-4a2d-85bf-cb75bdab2e35 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:09.176689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d35180-c3b4-4cc0-b5d5-62edb9b9588a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:09.177375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d35180-c3b4-4cc0-b5d5-62edb9b9588a service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:09.177660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d35180-c3b4-4cc0-b5d5-62edb9b9588a service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:09.177933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d35180-c3b4-4cc0-b5d5-62edb9b9588a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:09.180084 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-01d35180-c3b4-4cc0-b5d5-62edb9b9588a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:09.211228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:09.271294 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d35180-c3b4-4cc0-b5d5-62edb9b9588a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:09.273408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d35180-c3b4-4cc0-b5d5-62edb9b9588a service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:09.292334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 697/1400] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 07:58:11.652578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc921135-5cf7-4028-acac-cfad1bce95c7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:11.653114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc921135-5cf7-4028-acac-cfad1bce95c7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:11.653215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc921135-5cf7-4028-acac-cfad1bce95c7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:11.659039 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 704/1401] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:58:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:11.661019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8e896-5569-466d-849e-3acc6a59f95f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:11.661433 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8e896-5569-466d-849e-3acc6a59f95f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:11.661675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8e896-5569-466d-849e-3acc6a59f95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:11.781363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fbc8e896-5569-466d-849e-3acc6a59f95f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:11.781363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fbc8e896-5569-466d-849e-3acc6a59f95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:11.881550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 698/1402] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:12.213738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc921135-5cf7-4028-acac-cfad1bce95c7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:12.215640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bc921135-5cf7-4028-acac-cfad1bce95c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:12.389341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc921135-5cf7-4028-acac-cfad1bce95c7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:12.462886 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:12.525297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169af662-0436-4286-bc9f-0dcdae2cd42a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:12.526212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169af662-0436-4286-bc9f-0dcdae2cd42a service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:12.526493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169af662-0436-4286-bc9f-0dcdae2cd42a service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:12.526794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169af662-0436-4286-bc9f-0dcdae2cd42a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:12.530326 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-169af662-0436-4286-bc9f-0dcdae2cd42a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:12.633515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169af662-0436-4286-bc9f-0dcdae2cd42a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:12.635330 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169af662-0436-4286-bc9f-0dcdae2cd42a service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:12.658861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 705/1403] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:15.014327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a73e4-98c3-47d9-8f56-b6aa9d4918d9 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:15.014879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a73e4-98c3-47d9-8f56-b6aa9d4918d9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:15.014879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a73e4-98c3-47d9-8f56-b6aa9d4918d9 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:15.017542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 699/1404] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:58:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:15.021647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940c2b2-d947-4a7c-a6ce-8d619232c202 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:15.022179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940c2b2-d947-4a7c-a6ce-8d619232c202 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:15.022284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940c2b2-d947-4a7c-a6ce-8d619232c202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:15.090977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7940c2b2-d947-4a7c-a6ce-8d619232c202 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:15.092223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7940c2b2-d947-4a7c-a6ce-8d619232c202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:15.111437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:15.199382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 706/1405] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:15.490620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a73e4-98c3-47d9-8f56-b6aa9d4918d9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:15.492867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eb5a73e4-98c3-47d9-8f56-b6aa9d4918d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:15.497059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:15.660508 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a73e4-98c3-47d9-8f56-b6aa9d4918d9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:15.757511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:15.798397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d662f5a-40bb-481c-8b1b-f26bcde3c411 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:15.799058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d662f5a-40bb-481c-8b1b-f26bcde3c411 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:15.799337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d662f5a-40bb-481c-8b1b-f26bcde3c411 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:15.799613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d662f5a-40bb-481c-8b1b-f26bcde3c411 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:15.801250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0d662f5a-40bb-481c-8b1b-f26bcde3c411 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:15.900068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d662f5a-40bb-481c-8b1b-f26bcde3c411 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:15.901688 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d662f5a-40bb-481c-8b1b-f26bcde3c411 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:15.921298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 700/1406] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:18.247469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13949d4e-8fa4-44ad-9a40-a41de1337dc8 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:18.248054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13949d4e-8fa4-44ad-9a40-a41de1337dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:18.248054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13949d4e-8fa4-44ad-9a40-a41de1337dc8 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:18.253209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 707/1407] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:58:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:18.254526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052ab6b-510f-456f-b592-a02f295fb186 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:18.255018 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052ab6b-510f-456f-b592-a02f295fb186 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:18.255191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052ab6b-510f-456f-b592-a02f295fb186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:18.329201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0052ab6b-510f-456f-b592-a02f295fb186 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:18.330838 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0052ab6b-510f-456f-b592-a02f295fb186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:18.379629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:18.439539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 701/1408] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 07:58:18.730873 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13949d4e-8fa4-44ad-9a40-a41de1337dc8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:18.732303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-13949d4e-8fa4-44ad-9a40-a41de1337dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:18.888703 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13949d4e-8fa4-44ad-9a40-a41de1337dc8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:18.891550 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:19.015021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4deaa2-d7a6-4ec9-b366-14884c9f74cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:19.015589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4deaa2-d7a6-4ec9-b366-14884c9f74cf service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:19.015870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4deaa2-d7a6-4ec9-b366-14884c9f74cf service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:19.016143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4deaa2-d7a6-4ec9-b366-14884c9f74cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:19.017754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9b4deaa2-d7a6-4ec9-b366-14884c9f74cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:19.119233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4deaa2-d7a6-4ec9-b366-14884c9f74cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:19.120880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4deaa2-d7a6-4ec9-b366-14884c9f74cf service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:19.140822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 708/1409] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:28.437946 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=25 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:29.138135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:33.437500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094864d8-b1dc-4fa2-b78b-bcf7c95e8175 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:33.438062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094864d8-b1dc-4fa2-b78b-bcf7c95e8175 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:33.438062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094864d8-b1dc-4fa2-b78b-bcf7c95e8175 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:33.439425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 702/1410] 10.222.0.59 () {54 vars in 918 bytes} [Fri Sep 16 07:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:33.444625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a151e1-b0a2-40f7-8842-1b918de7c3d2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:33.445029 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a151e1-b0a2-40f7-8842-1b918de7c3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:33.445173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a151e1-b0a2-40f7-8842-1b918de7c3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:33.519684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a0a151e1-b0a2-40f7-8842-1b918de7c3d2 None None] MFA Rules not processed for user `c2dca05dfe8b4af0a02ea93fbe29c556`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:33.520877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a0a151e1-b0a2-40f7-8842-1b918de7c3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:33.626496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 709/1411] 10.222.0.59 () {58 vars in 1015 bytes} [Fri Sep 16 07:58:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:33.640741 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6724563-5405-4fb4-a50b-14a55c135539 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:33.641058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6724563-5405-4fb4-a50b-14a55c135539 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:33.641340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6724563-5405-4fb4-a50b-14a55c135539 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:33.642656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 703/1412] 10.222.0.59 () {54 vars in 918 bytes} [Fri Sep 16 07:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:33.647506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3abdbe-34d7-493d-98a3-114f364da669 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:33.647822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3abdbe-34d7-493d-98a3-114f364da669 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:33.648100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3abdbe-34d7-493d-98a3-114f364da669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:33.721183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:33.724885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9b3abdbe-34d7-493d-98a3-114f364da669 None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:33.726109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9b3abdbe-34d7-493d-98a3-114f364da669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:33.843024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 710/1413] 10.222.0.59 () {58 vars in 1015 bytes} [Fri Sep 16 07:58:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 07:58:36.394664 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1f00-5a9c-4df1-a8e1-cd2088dc70dd None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:36.395252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1f00-5a9c-4df1-a8e1-cd2088dc70dd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:36.395252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1f00-5a9c-4df1-a8e1-cd2088dc70dd None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:36.396563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 07:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:36.402352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6d1fb-377b-497c-ae7d-1043a8da4c0f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:36.402801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6d1fb-377b-497c-ae7d-1043a8da4c0f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:36.402931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6d1fb-377b-497c-ae7d-1043a8da4c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:36.468157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c8f6d1fb-377b-497c-ae7d-1043a8da4c0f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:36.469362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c8f6d1fb-377b-497c-ae7d-1043a8da4c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:36.522559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:36.563974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 07:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:36.848908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd1f00-5a9c-4df1-a8e1-cd2088dc70dd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:36.850488 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c4dd1f00-5a9c-4df1-a8e1-cd2088dc70dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:36.995062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd1f00-5a9c-4df1-a8e1-cd2088dc70dd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:37.066343 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:37.105865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216c2cc-8ddb-4860-ab07-9778a00a05b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:37.106468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216c2cc-8ddb-4860-ab07-9778a00a05b8 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:37.106774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216c2cc-8ddb-4860-ab07-9778a00a05b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:37.107049 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216c2cc-8ddb-4860-ab07-9778a00a05b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:37.108647 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7216c2cc-8ddb-4860-ab07-9778a00a05b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:37.196932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216c2cc-8ddb-4860-ab07-9778a00a05b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:37.198523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216c2cc-8ddb-4860-ab07-9778a00a05b8 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:37.216298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 705/1416] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:38.351922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1145f6-366a-42a2-88b5-712e880ad9b9 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:38.352488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1145f6-366a-42a2-88b5-712e880ad9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:38.352488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1145f6-366a-42a2-88b5-712e880ad9b9 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:38.353878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 07:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:38.360123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9a95f-0f1e-4f24-a05b-c3fb973fa8ba None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:38.360588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9a95f-0f1e-4f24-a05b-c3fb973fa8ba None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:38.360687 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9a95f-0f1e-4f24-a05b-c3fb973fa8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:38.436473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-92a9a95f-0f1e-4f24-a05b-c3fb973fa8ba None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:38.437593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-92a9a95f-0f1e-4f24-a05b-c3fb973fa8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:38.464536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:38.532717 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 07:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:38.683040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1145f6-366a-42a2-88b5-712e880ad9b9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:38.684580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ad1145f6-366a-42a2-88b5-712e880ad9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:38.829351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1145f6-366a-42a2-88b5-712e880ad9b9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:38.868862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:38.939867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcdc3c1-5712-42fe-b916-7e779a14dab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:38.940478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdc3c1-5712-42fe-b916-7e779a14dab3 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:38.940759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdc3c1-5712-42fe-b916-7e779a14dab3 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:38.941031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdc3c1-5712-42fe-b916-7e779a14dab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:38.942672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fdcdc3c1-5712-42fe-b916-7e779a14dab3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:39.032419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcdc3c1-5712-42fe-b916-7e779a14dab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:39.033937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcdc3c1-5712-42fe-b916-7e779a14dab3 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:39.051752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 713/1419] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:40.172584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229bec-42db-4543-b4f3-98f4631c4e19 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:40.173151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229bec-42db-4543-b4f3-98f4631c4e19 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:40.173151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229bec-42db-4543-b4f3-98f4631c4e19 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:40.174461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 07:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:40.181328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59907a-ee86-4461-8b61-fab7e22f0102 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:40.181732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59907a-ee86-4461-8b61-fab7e22f0102 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:40.181877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59907a-ee86-4461-8b61-fab7e22f0102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:40.235661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:40.257429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8a59907a-ee86-4461-8b61-fab7e22f0102 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:40.258787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8a59907a-ee86-4461-8b61-fab7e22f0102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:40.354168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 07:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:40.507802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae229bec-42db-4543-b4f3-98f4631c4e19 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:40.509315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ae229bec-42db-4543-b4f3-98f4631c4e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:40.656639 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae229bec-42db-4543-b4f3-98f4631c4e19 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:40.669001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:40.770181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e88c2d-5e83-4710-844d-14caaf23bf10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:40.770773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e88c2d-5e83-4710-844d-14caaf23bf10 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:40.771050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e88c2d-5e83-4710-844d-14caaf23bf10 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:40.771317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e88c2d-5e83-4710-844d-14caaf23bf10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:40.772918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-50e88c2d-5e83-4710-844d-14caaf23bf10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:40.860263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e88c2d-5e83-4710-844d-14caaf23bf10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:40.861848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e88c2d-5e83-4710-844d-14caaf23bf10 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:40.879028 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 708/1422] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:41.980758 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f588274-0b11-4d6d-b462-8450de57b1ab None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:41.981326 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f588274-0b11-4d6d-b462-8450de57b1ab None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:41.981326 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f588274-0b11-4d6d-b462-8450de57b1ab None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:41.982795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 07:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:41.989064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffcacb-7028-4d7a-8bfa-e939dbc8a518 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:41.989525 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffcacb-7028-4d7a-8bfa-e939dbc8a518 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:41.989622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffcacb-7028-4d7a-8bfa-e939dbc8a518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:42.010049 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:42.065818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-72ffcacb-7028-4d7a-8bfa-e939dbc8a518 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:42.067066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-72ffcacb-7028-4d7a-8bfa-e939dbc8a518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:42.161350 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 07:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:42.310190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f588274-0b11-4d6d-b462-8450de57b1ab None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:42.311687 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0f588274-0b11-4d6d-b462-8450de57b1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:42.421070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:42.457334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f588274-0b11-4d6d-b462-8450de57b1ab None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:42.568819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc82df-6eec-4db3-a79b-f8694326ea87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:42.569486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc82df-6eec-4db3-a79b-f8694326ea87 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:42.569771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc82df-6eec-4db3-a79b-f8694326ea87 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:42.570044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc82df-6eec-4db3-a79b-f8694326ea87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:42.571680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-71fc82df-6eec-4db3-a79b-f8694326ea87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:42.647316 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:42.660321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc82df-6eec-4db3-a79b-f8694326ea87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:42.661895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc82df-6eec-4db3-a79b-f8694326ea87 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:42.680904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 716/1425] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:43.782838 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a1bbf-5a17-4e85-b1b1-7113a982f1cd None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:43.783270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a1bbf-5a17-4e85-b1b1-7113a982f1cd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:43.783388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a1bbf-5a17-4e85-b1b1-7113a982f1cd None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:43.784738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 07:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:43.790875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a590-a703-499d-be29-0182b65af250 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:43.791336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a590-a703-499d-be29-0182b65af250 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:43.791435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a590-a703-499d-be29-0182b65af250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:43.866691 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2b87a590-a703-499d-be29-0182b65af250 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:43.868365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2b87a590-a703-499d-be29-0182b65af250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:43.961739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 07:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:44.108461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a1bbf-5a17-4e85-b1b1-7113a982f1cd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:44.110006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-487a1bbf-5a17-4e85-b1b1-7113a982f1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:44.192440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:44.253043 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a1bbf-5a17-4e85-b1b1-7113a982f1cd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:44.365247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6a99d-6bf8-4af7-8696-c2878b83699f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:44.365848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a99d-6bf8-4af7-8696-c2878b83699f service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:44.366122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a99d-6bf8-4af7-8696-c2878b83699f service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:44.366390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a99d-6bf8-4af7-8696-c2878b83699f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:44.369034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-69e6a99d-6bf8-4af7-8696-c2878b83699f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:44.416365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:44.456349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6a99d-6bf8-4af7-8696-c2878b83699f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:44.457866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6a99d-6bf8-4af7-8696-c2878b83699f service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:44.475744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 711/1428] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:45.581041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b9df4-fca9-474b-89f5-3784909ab45b None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:45.581568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b9df4-fca9-474b-89f5-3784909ab45b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:45.581568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b9df4-fca9-474b-89f5-3784909ab45b None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:45.582867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 07:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:45.589212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ca9e7-3eed-405f-91d3-0f528195c741 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:45.589652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ca9e7-3eed-405f-91d3-0f528195c741 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:45.589768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ca9e7-3eed-405f-91d3-0f528195c741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:45.664473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6e0ca9e7-3eed-405f-91d3-0f528195c741 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:45.665664 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6e0ca9e7-3eed-405f-91d3-0f528195c741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:45.760560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 07:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:45.911751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b9df4-fca9-474b-89f5-3784909ab45b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:45.913246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-195b9df4-fca9-474b-89f5-3784909ab45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:45.962112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:46.058728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b9df4-fca9-474b-89f5-3784909ab45b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:46.170343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30504cd6-ad01-4c3a-9c17-85341a84e30d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:46.170989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30504cd6-ad01-4c3a-9c17-85341a84e30d service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:46.171272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30504cd6-ad01-4c3a-9c17-85341a84e30d service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:46.171547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30504cd6-ad01-4c3a-9c17-85341a84e30d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:46.173423 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-30504cd6-ad01-4c3a-9c17-85341a84e30d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:46.180318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:46.262979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30504cd6-ad01-4c3a-9c17-85341a84e30d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:46.264570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30504cd6-ad01-4c3a-9c17-85341a84e30d service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:46.282703 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 719/1431] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:47.396238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e921a9-e9c8-49e7-be44-bc3283e32ee5 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:47.396747 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e921a9-e9c8-49e7-be44-bc3283e32ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:47.396844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e921a9-e9c8-49e7-be44-bc3283e32ee5 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:47.398106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 07:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:47.404787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca48067-160e-4bee-abeb-dc42f90008f6 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:47.405243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca48067-160e-4bee-abeb-dc42f90008f6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:47.405374 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca48067-160e-4bee-abeb-dc42f90008f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:47.480135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6ca48067-160e-4bee-abeb-dc42f90008f6 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:47.481372 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6ca48067-160e-4bee-abeb-dc42f90008f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:47.547753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:47.575475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 07:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:47.724807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e921a9-e9c8-49e7-be44-bc3283e32ee5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:47.726441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-60e921a9-e9c8-49e7-be44-bc3283e32ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:47.744483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:47.871489 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e921a9-e9c8-49e7-be44-bc3283e32ee5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:47.959441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:47.985229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e211fe-d3ef-41c3-8473-bf67ca0e0db6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:47.985837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e211fe-d3ef-41c3-8473-bf67ca0e0db6 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:47.986119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e211fe-d3ef-41c3-8473-bf67ca0e0db6 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:47.986394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e211fe-d3ef-41c3-8473-bf67ca0e0db6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:47.988077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-15e211fe-d3ef-41c3-8473-bf67ca0e0db6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:48.076642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e211fe-d3ef-41c3-8473-bf67ca0e0db6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:48.078238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e211fe-d3ef-41c3-8473-bf67ca0e0db6 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:48.095792 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 714/1434] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:49.174941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f816ca-b9a0-46a6-a7a3-42375fdb21e3 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:49.175515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f816ca-b9a0-46a6-a7a3-42375fdb21e3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:49.175515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f816ca-b9a0-46a6-a7a3-42375fdb21e3 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:49.176913 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 07:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:49.182976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6984449-d66b-4d26-8c7a-47d030330dd2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:49.183366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6984449-d66b-4d26-8c7a-47d030330dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:49.183555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6984449-d66b-4d26-8c7a-47d030330dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:49.258227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d6984449-d66b-4d26-8c7a-47d030330dd2 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:49.259444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d6984449-d66b-4d26-8c7a-47d030330dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:49.301936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:49.356636 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 07:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:49.504326 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f816ca-b9a0-46a6-a7a3-42375fdb21e3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:49.505765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-80f816ca-b9a0-46a6-a7a3-42375fdb21e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:49.648566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f816ca-b9a0-46a6-a7a3-42375fdb21e3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:49.702249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:49.762276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0eaa88-5ca4-47fb-a06d-5161252a5197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:49.762853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eaa88-5ca4-47fb-a06d-5161252a5197 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:49.763136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eaa88-5ca4-47fb-a06d-5161252a5197 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:49.763410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eaa88-5ca4-47fb-a06d-5161252a5197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:49.765099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fc0eaa88-5ca4-47fb-a06d-5161252a5197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:49.970232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0eaa88-5ca4-47fb-a06d-5161252a5197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:49.971842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0eaa88-5ca4-47fb-a06d-5161252a5197 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:49.989879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 722/1437] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:51.118808 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3295991-8970-4b84-ae76-a98918616501 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:51.119355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3295991-8970-4b84-ae76-a98918616501 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:51.119355 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3295991-8970-4b84-ae76-a98918616501 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:51.120705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 07:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:51.126900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bf009-c763-47a9-a272-b8b3e7b19cd4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:51.127371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bf009-c763-47a9-a272-b8b3e7b19cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:51.127470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bf009-c763-47a9-a272-b8b3e7b19cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:51.202615 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3f6bf009-c763-47a9-a272-b8b3e7b19cd4 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:51.203867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3f6bf009-c763-47a9-a272-b8b3e7b19cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:51.205856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:51.299369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 07:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:51.449059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3295991-8970-4b84-ae76-a98918616501 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:51.450619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a3295991-8970-4b84-ae76-a98918616501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:51.594489 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3295991-8970-4b84-ae76-a98918616501 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:51.622925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:51.707270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754ef63-09e5-48d9-8441-db865c8c9dd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:51.707810 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754ef63-09e5-48d9-8441-db865c8c9dd5 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:51.708090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754ef63-09e5-48d9-8441-db865c8c9dd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:51.708363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754ef63-09e5-48d9-8441-db865c8c9dd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:51.709975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1754ef63-09e5-48d9-8441-db865c8c9dd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:51.799512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754ef63-09e5-48d9-8441-db865c8c9dd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:51.801131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754ef63-09e5-48d9-8441-db865c8c9dd5 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:51.818640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 717/1440] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:53.864026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0d7ee-69e7-4332-b908-1cbad678e61e None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:53.864626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0d7ee-69e7-4332-b908-1cbad678e61e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:53.864626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0d7ee-69e7-4332-b908-1cbad678e61e None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:53.865904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 07:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:53.871274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9468c5-367e-46b0-a901-452711ea02e6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:53.871683 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9468c5-367e-46b0-a901-452711ea02e6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:53.871812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9468c5-367e-46b0-a901-452711ea02e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:53.913067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:53.937780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bd9468c5-367e-46b0-a901-452711ea02e6 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:53.939252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bd9468c5-367e-46b0-a901-452711ea02e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:54.033598 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 07:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:54.315491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0d7ee-69e7-4332-b908-1cbad678e61e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:54.317019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-eea0d7ee-69e7-4332-b908-1cbad678e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:54.419227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0d7ee-69e7-4332-b908-1cbad678e61e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:54.451325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:54.493123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8593a-ae4a-4915-8141-84ac628cfc2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:54.493722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8593a-ae4a-4915-8141-84ac628cfc2f service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:54.493999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8593a-ae4a-4915-8141-84ac628cfc2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:54.494270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8593a-ae4a-4915-8141-84ac628cfc2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:54.495899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-33d8593a-ae4a-4915-8141-84ac628cfc2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:54.602991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8593a-ae4a-4915-8141-84ac628cfc2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:54.604348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8593a-ae4a-4915-8141-84ac628cfc2f service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:54.622737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 725/1443] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:58:57.085230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdd801-83a1-42a7-9ee6-cca0e09f751b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:57.085767 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdd801-83a1-42a7-9ee6-cca0e09f751b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:57.085767 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdd801-83a1-42a7-9ee6-cca0e09f751b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:57.087086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 07:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:58:57.092211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54511f-bf22-422c-bf2c-bb499cad9c6c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:57.092648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54511f-bf22-422c-bf2c-bb499cad9c6c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:57.092756 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54511f-bf22-422c-bf2c-bb499cad9c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:57.156889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5d54511f-bf22-422c-bf2c-bb499cad9c6c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:58:57.158082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5d54511f-bf22-422c-bf2c-bb499cad9c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:57.170537 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:57.253678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 07:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:58:57.539601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdd801-83a1-42a7-9ee6-cca0e09f751b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:58:57.541127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-69fdd801-83a1-42a7-9ee6-cca0e09f751b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:57.643046 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdd801-83a1-42a7-9ee6-cca0e09f751b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:58:57.656044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:58:57.715584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45891edc-3595-4e8b-8fca-72f23ee64a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:58:57.716191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45891edc-3595-4e8b-8fca-72f23ee64a69 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:58:57.716466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45891edc-3595-4e8b-8fca-72f23ee64a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:58:57.716735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45891edc-3595-4e8b-8fca-72f23ee64a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:58:57.718337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-45891edc-3595-4e8b-8fca-72f23ee64a69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:58:57.828181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45891edc-3595-4e8b-8fca-72f23ee64a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:58:57.829561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45891edc-3595-4e8b-8fca-72f23ee64a69 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:58:57.847903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 720/1446] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:00.435139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb93d65-1de2-4fa8-96c4-a70d52cf8763 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:00.435744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb93d65-1de2-4fa8-96c4-a70d52cf8763 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:00.435744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb93d65-1de2-4fa8-96c4-a70d52cf8763 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:00.437045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 07:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:00.442405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01055a22-5d71-4ad6-9681-3ae010c3b850 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:00.442898 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01055a22-5d71-4ad6-9681-3ae010c3b850 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:00.443008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01055a22-5d71-4ad6-9681-3ae010c3b850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:00.490789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:00.509059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-01055a22-5d71-4ad6-9681-3ae010c3b850 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:00.510282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-01055a22-5d71-4ad6-9681-3ae010c3b850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:00.605385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 07:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:00.890237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb93d65-1de2-4fa8-96c4-a70d52cf8763 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:00.891838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2eb93d65-1de2-4fa8-96c4-a70d52cf8763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:00.993222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb93d65-1de2-4fa8-96c4-a70d52cf8763 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:01.032356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:01.064763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5af1f-a5f6-4f83-ab4e-5310a134de99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:01.065364 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5af1f-a5f6-4f83-ab4e-5310a134de99 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:01.065640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5af1f-a5f6-4f83-ab4e-5310a134de99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:01.065911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5af1f-a5f6-4f83-ab4e-5310a134de99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:01.067569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e5b5af1f-a5f6-4f83-ab4e-5310a134de99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:01.177471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5af1f-a5f6-4f83-ab4e-5310a134de99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:01.179516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5af1f-a5f6-4f83-ab4e-5310a134de99 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:01.196932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 728/1449] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:03.451728 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56cdd9-5ed4-407f-b496-b6e31181fb3b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:03.452302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56cdd9-5ed4-407f-b496-b6e31181fb3b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:03.452302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56cdd9-5ed4-407f-b496-b6e31181fb3b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:03.453593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 07:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:03.458792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42785-26bc-4120-9bb9-fed5821ab960 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:03.459242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42785-26bc-4120-9bb9-fed5821ab960 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:03.459355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42785-26bc-4120-9bb9-fed5821ab960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:03.524817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b9d42785-26bc-4120-9bb9-fed5821ab960 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:03.526012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b9d42785-26bc-4120-9bb9-fed5821ab960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:03.545910 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:03.623504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 07:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:03.909811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56cdd9-5ed4-407f-b496-b6e31181fb3b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:03.911380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4e56cdd9-5ed4-407f-b496-b6e31181fb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:04.013447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56cdd9-5ed4-407f-b496-b6e31181fb3b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:04.033435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:04.086628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f167a-b3f3-4448-a96b-93df9becc0b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:04.087254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f167a-b3f3-4448-a96b-93df9becc0b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:04.087528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f167a-b3f3-4448-a96b-93df9becc0b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:04.087797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f167a-b3f3-4448-a96b-93df9becc0b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:04.089410 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c89f167a-b3f3-4448-a96b-93df9becc0b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:04.196995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f167a-b3f3-4448-a96b-93df9becc0b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:04.198408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f167a-b3f3-4448-a96b-93df9becc0b6 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:04.215588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 723/1452] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:06.391632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9be3-ab05-4f4d-95cd-09608997e928 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:06.392202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9be3-ab05-4f4d-95cd-09608997e928 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:06.392202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9be3-ab05-4f4d-95cd-09608997e928 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:06.393504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 07:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:06.398864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17cf71-8584-4558-8a8c-429265ccfcfb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:06.399254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17cf71-8584-4558-8a8c-429265ccfcfb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:06.399413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17cf71-8584-4558-8a8c-429265ccfcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:06.462431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:06.465077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7b17cf71-8584-4558-8a8c-429265ccfcfb None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:06.466288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7b17cf71-8584-4558-8a8c-429265ccfcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:06.562143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 07:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:06.844194 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f9be3-ab05-4f4d-95cd-09608997e928 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:06.845725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e17f9be3-ab05-4f4d-95cd-09608997e928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:06.947471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f9be3-ab05-4f4d-95cd-09608997e928 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:06.996724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:07.019982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2099c07-06f2-40ff-b61d-2677544b1701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:07.020580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099c07-06f2-40ff-b61d-2677544b1701 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:07.020857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099c07-06f2-40ff-b61d-2677544b1701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:07.021127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099c07-06f2-40ff-b61d-2677544b1701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:07.023265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e2099c07-06f2-40ff-b61d-2677544b1701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:07.130346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2099c07-06f2-40ff-b61d-2677544b1701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:07.132309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2099c07-06f2-40ff-b61d-2677544b1701 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:07.150302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 731/1455] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:09.363619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5a59d-08d0-4046-b88a-a0e5058c68d0 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:09.365725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5a59d-08d0-4046-b88a-a0e5058c68d0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:09.365725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5a59d-08d0-4046-b88a-a0e5058c68d0 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:09.365725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 07:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:09.370865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e716f-9c68-4c85-93af-db0436002cc3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:09.371269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e716f-9c68-4c85-93af-db0436002cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:09.371415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e716f-9c68-4c85-93af-db0436002cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:09.436953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-da5e716f-9c68-4c85-93af-db0436002cc3 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:09.438150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-da5e716f-9c68-4c85-93af-db0436002cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:09.467149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:09.533544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 07:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:09.824839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5a59d-08d0-4046-b88a-a0e5058c68d0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:09.826373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7da5a59d-08d0-4046-b88a-a0e5058c68d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:09.927177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5a59d-08d0-4046-b88a-a0e5058c68d0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:09.955929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:10.000578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8ec99-73d9-4cce-8178-53f9a300f374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:10.001114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ec99-73d9-4cce-8178-53f9a300f374 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:10.001390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ec99-73d9-4cce-8178-53f9a300f374 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:10.001659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ec99-73d9-4cce-8178-53f9a300f374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:10.003266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-94c8ec99-73d9-4cce-8178-53f9a300f374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:10.110505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8ec99-73d9-4cce-8178-53f9a300f374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:10.111955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8ec99-73d9-4cce-8178-53f9a300f374 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:10.129992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 726/1458] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:12.322204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e366071-aa64-44f5-ac73-aafad14fed43 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:12.322747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e366071-aa64-44f5-ac73-aafad14fed43 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:12.322747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e366071-aa64-44f5-ac73-aafad14fed43 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:12.324063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 07:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:12.329618 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444428e-91bf-4676-9246-a9c1cd5e3484 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:12.330050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444428e-91bf-4676-9246-a9c1cd5e3484 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:12.330195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444428e-91bf-4676-9246-a9c1cd5e3484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:12.397796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7444428e-91bf-4676-9246-a9c1cd5e3484 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:12.399074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7444428e-91bf-4676-9246-a9c1cd5e3484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:12.407456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:12.494386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 07:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:12.779153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e366071-aa64-44f5-ac73-aafad14fed43 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:12.780722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6e366071-aa64-44f5-ac73-aafad14fed43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:12.883538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e366071-aa64-44f5-ac73-aafad14fed43 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:12.937479 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:12.955588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f3d93-9a79-4772-b8df-01ca91d025aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:12.956200 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f3d93-9a79-4772-b8df-01ca91d025aa service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:12.960781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f3d93-9a79-4772-b8df-01ca91d025aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:12.960781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f3d93-9a79-4772-b8df-01ca91d025aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:12.960781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dc3f3d93-9a79-4772-b8df-01ca91d025aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:13.066399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f3d93-9a79-4772-b8df-01ca91d025aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:13.067822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f3d93-9a79-4772-b8df-01ca91d025aa service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:13.086629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 734/1461] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:15.266259 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4174588-e2e9-4aa5-952c-d561f441ea25 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:15.266889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4174588-e2e9-4aa5-952c-d561f441ea25 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:15.266889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4174588-e2e9-4aa5-952c-d561f441ea25 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:15.268172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 07:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:15.273385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f7c13-57dc-4610-879f-2531a45f8bc6 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:15.273770 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f7c13-57dc-4610-879f-2531a45f8bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:15.273928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f7c13-57dc-4610-879f-2531a45f8bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:15.338952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8e4f7c13-57dc-4610-879f-2531a45f8bc6 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:15.340190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8e4f7c13-57dc-4610-879f-2531a45f8bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:15.378713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:15.435930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 07:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:15.717505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4174588-e2e9-4aa5-952c-d561f441ea25 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:15.719096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e4174588-e2e9-4aa5-952c-d561f441ea25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:15.822218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4174588-e2e9-4aa5-952c-d561f441ea25 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:15.859008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:15.894166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee389b-6842-466d-911a-bde9226588c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:15.894796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee389b-6842-466d-911a-bde9226588c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:15.895076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee389b-6842-466d-911a-bde9226588c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:15.895364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee389b-6842-466d-911a-bde9226588c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:15.897695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2cee389b-6842-466d-911a-bde9226588c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:16.004350 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee389b-6842-466d-911a-bde9226588c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:16.005721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee389b-6842-466d-911a-bde9226588c3 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:16.024414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 729/1464] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:18.185666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bd28f-28e4-4c1d-a0ab-a650de7ea30f None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:18.186234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bd28f-28e4-4c1d-a0ab-a650de7ea30f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:18.186234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bd28f-28e4-4c1d-a0ab-a650de7ea30f None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:18.187591 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 07:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:18.193154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71408cf5-965e-49b5-935a-998ab61fa664 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:18.193572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71408cf5-965e-49b5-935a-998ab61fa664 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:18.193689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71408cf5-965e-49b5-935a-998ab61fa664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:18.259769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-71408cf5-965e-49b5-935a-998ab61fa664 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:18.260971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-71408cf5-965e-49b5-935a-998ab61fa664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:18.279106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:18.357150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 07:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:18.636458 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bd28f-28e4-4c1d-a0ab-a650de7ea30f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:18.637996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ec7bd28f-28e4-4c1d-a0ab-a650de7ea30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:18.741345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bd28f-28e4-4c1d-a0ab-a650de7ea30f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:18.807579 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:18.813931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fca472-f73c-49bc-9f85-988fcb1ab9b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:18.814538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fca472-f73c-49bc-9f85-988fcb1ab9b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:18.814851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fca472-f73c-49bc-9f85-988fcb1ab9b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:18.815124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fca472-f73c-49bc-9f85-988fcb1ab9b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:18.816719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-94fca472-f73c-49bc-9f85-988fcb1ab9b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:18.923365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fca472-f73c-49bc-9f85-988fcb1ab9b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:18.924776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fca472-f73c-49bc-9f85-988fcb1ab9b8 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:18.942545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 737/1467] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:21.063824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22083-8cb7-4e4e-9701-c810a7b357ca None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:21.064398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22083-8cb7-4e4e-9701-c810a7b357ca None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:21.064398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22083-8cb7-4e4e-9701-c810a7b357ca None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:21.065692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 07:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:21.070880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85893f-b0f9-43a3-8e93-2d863f51c332 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:21.071297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85893f-b0f9-43a3-8e93-2d863f51c332 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:21.071427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85893f-b0f9-43a3-8e93-2d863f51c332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:21.136684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1d85893f-b0f9-43a3-8e93-2d863f51c332 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:21.137899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1d85893f-b0f9-43a3-8e93-2d863f51c332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:21.186851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:21.240658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 07:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:21.525748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f22083-8cb7-4e4e-9701-c810a7b357ca None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:21.527316 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-72f22083-8cb7-4e4e-9701-c810a7b357ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:21.630479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f22083-8cb7-4e4e-9701-c810a7b357ca None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:21.677984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:21.703792 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf578704-f510-4a55-993c-e45d3cc64876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:21.704394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf578704-f510-4a55-993c-e45d3cc64876 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:21.704667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf578704-f510-4a55-993c-e45d3cc64876 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:21.704937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf578704-f510-4a55-993c-e45d3cc64876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:21.706530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cf578704-f510-4a55-993c-e45d3cc64876 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:21.813003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf578704-f510-4a55-993c-e45d3cc64876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:21.814434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf578704-f510-4a55-993c-e45d3cc64876 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:21.832856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 732/1470] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:31.239141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=22 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:31.830601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=66 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:41.576590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba58e7-b4fc-4282-8778-9c74f696d8da None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:41.577162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba58e7-b4fc-4282-8778-9c74f696d8da None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:41.577162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba58e7-b4fc-4282-8778-9c74f696d8da None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:41.578489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 739/1471] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:41.583645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6121f-8604-4ad8-9b3c-834216f72cb2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:41.584102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6121f-8604-4ad8-9b3c-834216f72cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:41.584210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6121f-8604-4ad8-9b3c-834216f72cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:41.650213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-48b6121f-8604-4ad8-9b3c-834216f72cb2 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:41.651362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-48b6121f-8604-4ad8-9b3c-834216f72cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:41.748723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 733/1472] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:42.047534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38797b27-8226-478b-9a73-6f61da667e9a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:42.047817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38797b27-8226-478b-9a73-6f61da667e9a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:42.048127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38797b27-8226-478b-9a73-6f61da667e9a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:42.049492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 740/1473] 10.222.0.59 () {54 vars in 962 bytes} [Fri Sep 16 07:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:42.054458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50af8a4-f59b-4c94-a832-0009d1c839b0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:42.054775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50af8a4-f59b-4c94-a832-0009d1c839b0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:42.055055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50af8a4-f59b-4c94-a832-0009d1c839b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:42.129553 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e50af8a4-f59b-4c94-a832-0009d1c839b0 None None] MFA Rules not processed for user `c214b2c97a7b41698b433eac8379442a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:42.130780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e50af8a4-f59b-4c94-a832-0009d1c839b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:42.150998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:42.205303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 734/1474] 10.222.0.59 () {58 vars in 1059 bytes} [Fri Sep 16 07:59:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:42.213713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38797b27-8226-478b-9a73-6f61da667e9a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:42.215265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-38797b27-8226-478b-9a73-6f61da667e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:42.316422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38797b27-8226-478b-9a73-6f61da667e9a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:42.388732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7bcd9-8b67-4197-93c3-f098e4f511de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c214b2c97a7b41698b433eac8379442a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:42.389367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7bcd9-8b67-4197-93c3-f098e4f511de service glance] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:42.389641 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7bcd9-8b67-4197-93c3-f098e4f511de service glance] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:42.389988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7bcd9-8b67-4197-93c3-f098e4f511de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:42.391556 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6bf7bcd9-8b67-4197-93c3-f098e4f511de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:42.445533 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:42.500044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7bcd9-8b67-4197-93c3-f098e4f511de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:42.501500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7bcd9-8b67-4197-93c3-f098e4f511de service glance] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:42.521322 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 741/1475] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:44.725376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dda30d-f71d-4bad-8190-9aada02efb62 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:44.725949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dda30d-f71d-4bad-8190-9aada02efb62 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:44.725949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dda30d-f71d-4bad-8190-9aada02efb62 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:44.727289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 735/1476] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:44.732514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786f5fa-ddb3-4a46-9d46-bf42c8ee92b1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:44.732911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786f5fa-ddb3-4a46-9d46-bf42c8ee92b1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:44.733056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786f5fa-ddb3-4a46-9d46-bf42c8ee92b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:44.798119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2786f5fa-ddb3-4a46-9d46-bf42c8ee92b1 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:44.799361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2786f5fa-ddb3-4a46-9d46-bf42c8ee92b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:44.893948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 742/1477] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:45.163124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dda30d-f71d-4bad-8190-9aada02efb62 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:45.164628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f4dda30d-f71d-4bad-8190-9aada02efb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:45.288899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dda30d-f71d-4bad-8190-9aada02efb62 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:45.338315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:45.381751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7fa6f-a48d-44ac-b97d-eb1e8e93d421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:45.382356 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7fa6f-a48d-44ac-b97d-eb1e8e93d421 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:45.382637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7fa6f-a48d-44ac-b97d-eb1e8e93d421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:45.382943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7fa6f-a48d-44ac-b97d-eb1e8e93d421 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:45.383579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7fa6f-a48d-44ac-b97d-eb1e8e93d421 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:45.385357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-3aa7fa6f-a48d-44ac-b97d-eb1e8e93d421 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 07:59:45.385357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 16 07:59:45.385357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: Sep 16 07:59:45.386598 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7fa6f-a48d-44ac-b97d-eb1e8e93d421 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:45.407264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 736/1478] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Sep 16 07:59:45 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 07:59:47.526848 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e9968-8cff-4ef4-ad23-6427a577a54a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:47.527417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e9968-8cff-4ef4-ad23-6427a577a54a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:47.527417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e9968-8cff-4ef4-ad23-6427a577a54a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:47.528766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 743/1479] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:47.534210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fd12b-e9d6-4f58-b2fc-893573ab2d2f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:47.534635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fd12b-e9d6-4f58-b2fc-893573ab2d2f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:47.534819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fd12b-e9d6-4f58-b2fc-893573ab2d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:47.601182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-633fd12b-e9d6-4f58-b2fc-893573ab2d2f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:47.602464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-633fd12b-e9d6-4f58-b2fc-893573ab2d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:47.696581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 737/1480] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:47.990609 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e9968-8cff-4ef4-ad23-6427a577a54a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:47.992226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2b5e9968-8cff-4ef4-ad23-6427a577a54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:48.001812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:48.136708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e9968-8cff-4ef4-ad23-6427a577a54a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:48.216562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:48.249206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8d99b-82da-4c6c-9a0c-ca3d1ab76579 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:48.249750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8d99b-82da-4c6c-9a0c-ca3d1ab76579 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:48.250034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8d99b-82da-4c6c-9a0c-ca3d1ab76579 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:48.250306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8d99b-82da-4c6c-9a0c-ca3d1ab76579 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:48.251954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d8d8d99b-82da-4c6c-9a0c-ca3d1ab76579 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:48.341202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8d99b-82da-4c6c-9a0c-ca3d1ab76579 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:48.342837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8d99b-82da-4c6c-9a0c-ca3d1ab76579 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:48.360835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 744/1481] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:50.668630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d30b2-b652-4055-9821-6709b5993434 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:50.669203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d30b2-b652-4055-9821-6709b5993434 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:50.669203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d30b2-b652-4055-9821-6709b5993434 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:50.670803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 738/1482] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:50.676069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce12bd8-20d3-490f-a3c7-806b50a06c24 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:50.676486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce12bd8-20d3-490f-a3c7-806b50a06c24 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:50.676627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce12bd8-20d3-490f-a3c7-806b50a06c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:50.743248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9ce12bd8-20d3-490f-a3c7-806b50a06c24 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:50.744531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9ce12bd8-20d3-490f-a3c7-806b50a06c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:50.790628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:50.840489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 745/1483] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:51.128400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d30b2-b652-4055-9821-6709b5993434 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:51.129985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7f1d30b2-b652-4055-9821-6709b5993434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:51.154189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:51.232788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d30b2-b652-4055-9821-6709b5993434 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:51.304156 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045a5c2-dc26-456b-be57-eb44441ec9bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:51.304758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045a5c2-dc26-456b-be57-eb44441ec9bf service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:51.305037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045a5c2-dc26-456b-be57-eb44441ec9bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:51.305307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045a5c2-dc26-456b-be57-eb44441ec9bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:51.306956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9045a5c2-dc26-456b-be57-eb44441ec9bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:51.383362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:51.419414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045a5c2-dc26-456b-be57-eb44441ec9bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:51.421136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045a5c2-dc26-456b-be57-eb44441ec9bf service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:51.438958 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 739/1484] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:53.630550 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b57cab-1ff4-4e72-acca-ea897705a0d7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:53.631118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b57cab-1ff4-4e72-acca-ea897705a0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:53.631118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b57cab-1ff4-4e72-acca-ea897705a0d7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:53.632426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 746/1485] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:53.637689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9400-cc48-4986-a1d3-63fa275f37e9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:53.638147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9400-cc48-4986-a1d3-63fa275f37e9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:53.638243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9400-cc48-4986-a1d3-63fa275f37e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:53.704961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-aabf9400-cc48-4986-a1d3-63fa275f37e9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:53.706239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-aabf9400-cc48-4986-a1d3-63fa275f37e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:53.803761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 740/1486] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:54.089905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b57cab-1ff4-4e72-acca-ea897705a0d7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:54.091527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-86b57cab-1ff4-4e72-acca-ea897705a0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:54.192539 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b57cab-1ff4-4e72-acca-ea897705a0d7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:54.262960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:54.264395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9f45f-bd7e-4d28-85a7-8f9deeee558a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:54.264991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9f45f-bd7e-4d28-85a7-8f9deeee558a service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:54.265268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9f45f-bd7e-4d28-85a7-8f9deeee558a service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:54.265536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9f45f-bd7e-4d28-85a7-8f9deeee558a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:54.267138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-22d9f45f-bd7e-4d28-85a7-8f9deeee558a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:54.374181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9f45f-bd7e-4d28-85a7-8f9deeee558a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:54.375599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9f45f-bd7e-4d28-85a7-8f9deeee558a service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:54.394235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 747/1487] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 07:59:56.784885 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efca62-b627-48b5-95dd-5870742a0fea None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:56.785452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efca62-b627-48b5-95dd-5870742a0fea None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:56.785452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efca62-b627-48b5-95dd-5870742a0fea None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:56.787351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 741/1488] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 07:59:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 07:59:56.792784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5536127-b877-4589-9d0f-7b9244b9aaaa None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:56.793196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5536127-b877-4589-9d0f-7b9244b9aaaa None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:56.793338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5536127-b877-4589-9d0f-7b9244b9aaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:56.859769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b5536127-b877-4589-9d0f-7b9244b9aaaa None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 07:59:56.860978 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b5536127-b877-4589-9d0f-7b9244b9aaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:56.913158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:56.956295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 748/1489] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 07:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 07:59:57.242908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efca62-b627-48b5-95dd-5870742a0fea None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 07:59:57.244517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f6efca62-b627-48b5-95dd-5870742a0fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:57.280408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:57.348262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efca62-b627-48b5-95dd-5870742a0fea None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 07:59:57.422474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565ed63-0c9b-4e53-989b-6bd7f70a105e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 07:59:57.423109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ed63-0c9b-4e53-989b-6bd7f70a105e service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 07:59:57.423388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ed63-0c9b-4e53-989b-6bd7f70a105e service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 07:59:57.423659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565ed63-0c9b-4e53-989b-6bd7f70a105e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 07:59:57.425258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0565ed63-0c9b-4e53-989b-6bd7f70a105e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 07:59:57.503149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 07:59:57.531763 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565ed63-0c9b-4e53-989b-6bd7f70a105e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 07:59:57.533246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565ed63-0c9b-4e53-989b-6bd7f70a105e service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 07:59:57.551351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 742/1490] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 07:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:00:00.032245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00640f-3e2c-4b4f-94ba-e850a21b20cc None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:00:00.032765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00640f-3e2c-4b4f-94ba-e850a21b20cc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:00:00.032765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00640f-3e2c-4b4f-94ba-e850a21b20cc None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:00:00.034089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 749/1491] 10.222.0.59 () {54 vars in 927 bytes} [Fri Sep 16 08:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:00:00.040087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ae83d-cccb-4f33-b281-e970a752289b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:00:00.040491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ae83d-cccb-4f33-b281-e970a752289b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:00:00.040635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ae83d-cccb-4f33-b281-e970a752289b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:00:00.106933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a03ae83d-cccb-4f33-b281-e970a752289b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:00:00.108187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a03ae83d-cccb-4f33-b281-e970a752289b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:00:00.202443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 743/1492] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Sep 16 08:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:00:00.491220 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00640f-3e2c-4b4f-94ba-e850a21b20cc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:00:00.492771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5e00640f-3e2c-4b4f-94ba-e850a21b20cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:00:00.595504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00640f-3e2c-4b4f-94ba-e850a21b20cc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:00:00.669873 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bda3a6-5025-4d66-bfa1-85f3431194af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:00:00.670775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bda3a6-5025-4d66-bfa1-85f3431194af service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:00:00.671056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bda3a6-5025-4d66-bfa1-85f3431194af service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:00:00.671331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bda3a6-5025-4d66-bfa1-85f3431194af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:00:00.672957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-82bda3a6-5025-4d66-bfa1-85f3431194af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:00:00.681484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:00:00.784260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bda3a6-5025-4d66-bfa1-85f3431194af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:00:00.785717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bda3a6-5025-4d66-bfa1-85f3431194af service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:00:00.804066 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 750/1493] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:00:10.201401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=82 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:00:10.801926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=50 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:01:45.202899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a6027-94f7-4195-b20e-5dd4b810b8d8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:45.203496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a6027-94f7-4195-b20e-5dd4b810b8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:45.203496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a6027-94f7-4195-b20e-5dd4b810b8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:45.282067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c50a6027-94f7-4195-b20e-5dd4b810b8d8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:01:45.283287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c50a6027-94f7-4195-b20e-5dd4b810b8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:01:45.384611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 744/1494] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:01:45.392444 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c130e-2042-43a4-a0bc-755df3db1a5b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:45.392921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c130e-2042-43a4-a0bc-755df3db1a5b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:45.393025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c130e-2042-43a4-a0bc-755df3db1a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:45.412315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e45c130e-2042-43a4-a0bc-755df3db1a5b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:01:45.413502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e45c130e-2042-43a4-a0bc-755df3db1a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:01:45.422668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 751/1495] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:01:45.427421 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a6027-94f7-4195-b20e-5dd4b810b8d8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:01:45.436961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a6027-94f7-4195-b20e-5dd4b810b8d8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:01:45.442289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153d4eb-3efc-4d72-b964-cd1f426af03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:01:45.442912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153d4eb-3efc-4d72-b964-cd1f426af03c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:45.443201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153d4eb-3efc-4d72-b964-cd1f426af03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:45.443474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153d4eb-3efc-4d72-b964-cd1f426af03c admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:45.444152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153d4eb-3efc-4d72-b964-cd1f426af03c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:01:45.444456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153d4eb-3efc-4d72-b964-cd1f426af03c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:01:45.445473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-3153d4eb-3efc-4d72-b964-cd1f426af03c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:01:45.445473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 16 08:01:45.445473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: Sep 16 08:01:45.446356 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153d4eb-3efc-4d72-b964-cd1f426af03c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:01:45.457608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 745/1496] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:01:45.465374 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c130e-2042-43a4-a0bc-755df3db1a5b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:01:45.474303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c130e-2042-43a4-a0bc-755df3db1a5b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:01:45.478834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d429604-e65e-400e-9486-d9fb328e0f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:01:45.479420 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d429604-e65e-400e-9486-d9fb328e0f78 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:45.479698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d429604-e65e-400e-9486-d9fb328e0f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:45.479968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d429604-e65e-400e-9486-d9fb328e0f78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:45.480590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d429604-e65e-400e-9486-d9fb328e0f78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:01:45.482301 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-4d429604-e65e-400e-9486-d9fb328e0f78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:01:45.482301 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 16 08:01:45.482301 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: Sep 16 08:01:45.483517 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d429604-e65e-400e-9486-d9fb328e0f78 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:01:45.591691 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 752/1497] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:01:45 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:01:45.595931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153d4eb-3efc-4d72-b964-cd1f426af03c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:01:45.605823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153d4eb-3efc-4d72-b964-cd1f426af03c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:01:45.655456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:01:45.662644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876f010-fbf4-44ec-9a47-d1909397e15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:01:45.663860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876f010-fbf4-44ec-9a47-d1909397e15a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:45.663860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876f010-fbf4-44ec-9a47-d1909397e15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:45.663860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876f010-fbf4-44ec-9a47-d1909397e15a admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:45.664559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876f010-fbf4-44ec-9a47-d1909397e15a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:01:45.665612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-0876f010-fbf4-44ec-9a47-d1909397e15a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:01:45.665612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 16 08:01:45.665612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: Sep 16 08:01:45.666872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876f010-fbf4-44ec-9a47-d1909397e15a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:01:45.740346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 746/1498] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:01:45 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:01:45.744368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d429604-e65e-400e-9486-d9fb328e0f78 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:01:45.754603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d429604-e65e-400e-9486-d9fb328e0f78 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:01:45.759234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01357fec-5cc4-4b3d-b676-c0e7cb01be06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:01:45.759828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01357fec-5cc4-4b3d-b676-c0e7cb01be06 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:45.760106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01357fec-5cc4-4b3d-b676-c0e7cb01be06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:45.760419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01357fec-5cc4-4b3d-b676-c0e7cb01be06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:45.761086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01357fec-5cc4-4b3d-b676-c0e7cb01be06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:01:45.762106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-01357fec-5cc4-4b3d-b676-c0e7cb01be06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:01:45.762106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 16 08:01:45.762106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: Sep 16 08:01:45.763015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01357fec-5cc4-4b3d-b676-c0e7cb01be06 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:01:45.775560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 753/1499] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:01:45.779466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876f010-fbf4-44ec-9a47-d1909397e15a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:01:45.789887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876f010-fbf4-44ec-9a47-d1909397e15a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:01:45.794835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b28dc6-0850-4ee3-836d-d48c76b3c83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:01:45.795450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b28dc6-0850-4ee3-836d-d48c76b3c83d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:45.795735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b28dc6-0850-4ee3-836d-d48c76b3c83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:45.796012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b28dc6-0850-4ee3-836d-d48c76b3c83d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:45.796654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b28dc6-0850-4ee3-836d-d48c76b3c83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:01:45.797669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-01b28dc6-0850-4ee3-836d-d48c76b3c83d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:01:45.797669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 16 08:01:45.797669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: Sep 16 08:01:45.798543 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b28dc6-0850-4ee3-836d-d48c76b3c83d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:01:45.812297 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 747/1500] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:01:45.816276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01357fec-5cc4-4b3d-b676-c0e7cb01be06 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:01:45.825994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01357fec-5cc4-4b3d-b676-c0e7cb01be06 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:01:45.830571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51838db-0452-4810-b94b-3cd82fe2ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:01:45.831249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51838db-0452-4810-b94b-3cd82fe2ce0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:45.831529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51838db-0452-4810-b94b-3cd82fe2ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:45.831805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51838db-0452-4810-b94b-3cd82fe2ce0f admin admin] PATH_INFO: `/v3/projects/c8f97896b2ca452495267cf4fbc2bc93/users/d86726ca1e7a4810a0298492ca47ab21/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:45.845906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51838db-0452-4810-b94b-3cd82fe2ce0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f97896b2ca452495267cf4fbc2bc93, user_id=d86726ca1e7a4810a0298492ca47ab21, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:01:45.846960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-b51838db-0452-4810-b94b-3cd82fe2ce0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:01:45.846960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 16 08:01:45.846960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: Sep 16 08:01:45.849380 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51838db-0452-4810-b94b-3cd82fe2ce0f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:01:45.910741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 754/1501] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:01:45 2022] PUT /identity/v3/projects/c8f97896b2ca452495267cf4fbc2bc93/users/d86726ca1e7a4810a0298492ca47ab21/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:01:45.928713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211f60-aea6-403e-b2dc-0b7799f83cd5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:45.929023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211f60-aea6-403e-b2dc-0b7799f83cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:45.929308 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211f60-aea6-403e-b2dc-0b7799f83cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:45.956440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9b211f60-aea6-403e-b2dc-0b7799f83cd5 None None] MFA Rules not processed for user `d86726ca1e7a4810a0298492ca47ab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:01:45.957647 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9b211f60-aea6-403e-b2dc-0b7799f83cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:01:46.019898 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 748/1502] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:01:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:01:46.032935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173864c7-663c-4ea0-a293-6fd008841c33 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:46.033217 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173864c7-663c-4ea0-a293-6fd008841c33 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:46.033486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173864c7-663c-4ea0-a293-6fd008841c33 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:46.034839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 755/1503] 10.222.0.59 () {54 vars in 964 bytes} [Fri Sep 16 08:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:01:46.040077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d068d8-b00d-4617-b163-92c2e6b54bae None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:46.040363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d068d8-b00d-4617-b163-92c2e6b54bae None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:46.040637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d068d8-b00d-4617-b163-92c2e6b54bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:46.098709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-84d068d8-b00d-4617-b163-92c2e6b54bae None None] MFA Rules not processed for user `fb514134abe54e5ca8125e572a2ff928`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:01:46.099906 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-84d068d8-b00d-4617-b163-92c2e6b54bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:01:46.173317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 749/1504] 10.222.0.59 () {58 vars in 1061 bytes} [Fri Sep 16 08:01:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:01:46.186148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173864c7-663c-4ea0-a293-6fd008841c33 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:01:46.196634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173864c7-663c-4ea0-a293-6fd008841c33 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:01:46.200069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f8086-b1c5-4f47-8db9-bad9a206f5ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:01:46.200655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f8086-b1c5-4f47-8db9-bad9a206f5ee service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:46.200931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f8086-b1c5-4f47-8db9-bad9a206f5ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:46.201199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f8086-b1c5-4f47-8db9-bad9a206f5ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:46.209614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f8086-b1c5-4f47-8db9-bad9a206f5ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:01:46.211083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f8086-b1c5-4f47-8db9-bad9a206f5ee service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:01:46.219542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 756/1505] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:01:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:01:46.264079 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532a476-ae26-4ff4-8d42-0bab9f8132ad None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:46.264511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532a476-ae26-4ff4-8d42-0bab9f8132ad None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:46.264658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532a476-ae26-4ff4-8d42-0bab9f8132ad None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:46.265929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 750/1506] 10.222.0.59 () {52 vars in 830 bytes} [Fri Sep 16 08:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:01:46.270069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f8086-b1c5-4f47-8db9-bad9a206f5ee service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:01:46.331363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f8086-b1c5-4f47-8db9-bad9a206f5ee service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:01:46.336406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:01:46.337032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:46.337315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:46.337592 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] PATH_INFO: `/v3/users/d86726ca1e7a4810a0298492ca47ab21` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:46.338834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86726ca1e7a4810a0298492ca47ab21)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:01:46.339857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:01:46.339857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 16 08:01:46.339857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: Sep 16 08:01:46.341093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:01:46.548313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86726ca1e7a4810a0298492ca47ab21'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:01:46.572733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86726ca1e7a4810a0298492ca47ab21'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:01:46.587543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86726ca1e7a4810a0298492ca47ab21'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:01:46.592117 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:01:46.664746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86726ca1e7a4810a0298492ca47ab21'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:01:46.665382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86726ca1e7a4810a0298492ca47ab21'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:01:46.672195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 757/1507] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:01:46 2022] DELETE /identity/v3/users/d86726ca1e7a4810a0298492ca47ab21 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:01:46.676702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e68f52-5119-420e-9262-46e8e66b0d81 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:46.676984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e68f52-5119-420e-9262-46e8e66b0d81 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:46.677285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e68f52-5119-420e-9262-46e8e66b0d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:46.696525 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-77e68f52-5119-420e-9262-46e8e66b0d81 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:01:46.697730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-77e68f52-5119-420e-9262-46e8e66b0d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:01:46.706799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:01:46.757699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 751/1508] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:01:46.767375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:01:46.768776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:01:46.845854 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6fbd4-0857-4d96-b2b2-923bcbc3dd06 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:01:46.849382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15648d2e-9563-44ed-8bfe-da3b79e2bbda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:01:46.849984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15648d2e-9563-44ed-8bfe-da3b79e2bbda service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:46.850264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15648d2e-9563-44ed-8bfe-da3b79e2bbda service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:46.850537 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15648d2e-9563-44ed-8bfe-da3b79e2bbda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:46.859189 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15648d2e-9563-44ed-8bfe-da3b79e2bbda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:01:46.860615 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15648d2e-9563-44ed-8bfe-da3b79e2bbda service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:01:46.870213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 758/1509] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:01:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:01:47.318001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e68f52-5119-420e-9262-46e8e66b0d81 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:01:47.319577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-77e68f52-5119-420e-9262-46e8e66b0d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:01:47.332159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e68f52-5119-420e-9262-46e8e66b0d81 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:01:47.336944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:01:47.337578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:01:47.337858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:01:47.338130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] PATH_INFO: `/v3/projects/c8f97896b2ca452495267cf4fbc2bc93` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:01:47.339293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f97896b2ca452495267cf4fbc2bc93)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:01:47.340463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:01:47.340463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 16 08:01:47.340463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: Sep 16 08:01:47.341567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:01:47.453310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f97896b2ca452495267cf4fbc2bc93', 'request_id': 'req-c0710656-b17a-4313-8517-e2602849e383'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:01:47.467065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f97896b2ca452495267cf4fbc2bc93', 'request_id': 'req-c0710656-b17a-4313-8517-e2602849e383'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:01:47.473997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f97896b2ca452495267cf4fbc2bc93', 'request_id': 'req-c0710656-b17a-4313-8517-e2602849e383'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:01:47.539207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] The token cache is being invalidate because project c8f97896b2ca452495267cf4fbc2bc93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:01:47.539500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:01:47.540153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c0710656-b17a-4313-8517-e2602849e383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:01:47.541750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 752/1510] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:01:47 2022] DELETE /identity/v3/projects/c8f97896b2ca452495267cf4fbc2bc93 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:01:56.866237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:01:57.535876 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:03:14.685697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b919e-bbf0-403c-9c2f-2b273a4a7a37 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:14.686339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b919e-bbf0-403c-9c2f-2b273a4a7a37 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:14.686339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b919e-bbf0-403c-9c2f-2b273a4a7a37 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:14.687687 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 08:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:03:14.693329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb16c1-f639-4a8b-b659-1ebd737e0ebc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:14.693778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb16c1-f639-4a8b-b659-1ebd737e0ebc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:14.693881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb16c1-f639-4a8b-b659-1ebd737e0ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:14.713912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-30cb16c1-f639-4a8b-b659-1ebd737e0ebc None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:03:14.715180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-30cb16c1-f639-4a8b-b659-1ebd737e0ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:14.773851 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 08:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:03:14.925951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b919e-bbf0-403c-9c2f-2b273a4a7a37 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:03:14.927823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c96b919e-bbf0-403c-9c2f-2b273a4a7a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:15.028920 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b919e-bbf0-403c-9c2f-2b273a4a7a37 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:03:15.034336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f88fe-334b-4935-80b4-17d22cd0ba73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:03:15.034975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f88fe-334b-4935-80b4-17d22cd0ba73 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:15.035259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f88fe-334b-4935-80b4-17d22cd0ba73 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:15.035533 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f88fe-334b-4935-80b4-17d22cd0ba73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:15.043732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f88fe-334b-4935-80b4-17d22cd0ba73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:03:15.045379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f88fe-334b-4935-80b4-17d22cd0ba73 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:03:15.055143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 760/1513] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:03:16.129970 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8602a3-8e6c-451c-899f-6220578dfb3c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:16.130522 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8602a3-8e6c-451c-899f-6220578dfb3c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:16.130522 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8602a3-8e6c-451c-899f-6220578dfb3c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:16.131820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 08:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:03:16.137776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96979e88-06a4-41eb-9db6-f01803419e41 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:16.138226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96979e88-06a4-41eb-9db6-f01803419e41 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:16.138327 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96979e88-06a4-41eb-9db6-f01803419e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:16.160282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-96979e88-06a4-41eb-9db6-f01803419e41 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:03:16.161496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-96979e88-06a4-41eb-9db6-f01803419e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:16.169756 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 08:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:03:16.319866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b9c0f-39fc-40b9-a7b3-f799e434c16a None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:16.320154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b9c0f-39fc-40b9-a7b3-f799e434c16a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:16.320432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b9c0f-39fc-40b9-a7b3-f799e434c16a None None] PATH_INFO: `/v3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:16.321704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 755/1516] 10.222.0.42 () {54 vars in 925 bytes} [Fri Sep 16 08:03:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:03:16.324021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96979e88-06a4-41eb-9db6-f01803419e41 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:03:16.336829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96979e88-06a4-41eb-9db6-f01803419e41 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:03:16.342096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91311ad6-42fe-4b2f-adfb-a8d5939409dc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:16.342378 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91311ad6-42fe-4b2f-adfb-a8d5939409dc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:16.342554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4cfa0-b1db-447f-b18f-f9bc273dd9c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:03:16.342778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91311ad6-42fe-4b2f-adfb-a8d5939409dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:16.343191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4cfa0-b1db-447f-b18f-f9bc273dd9c2 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:16.343472 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4cfa0-b1db-447f-b18f-f9bc273dd9c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:16.343748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4cfa0-b1db-447f-b18f-f9bc273dd9c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:16.351976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4cfa0-b1db-447f-b18f-f9bc273dd9c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:03:16.353565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4cfa0-b1db-447f-b18f-f9bc273dd9c2 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:03:16.363541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 762/1517] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:03:16.395251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-91311ad6-42fe-4b2f-adfb-a8d5939409dc None None] MFA Rules not processed for user `c2dca05dfe8b4af0a02ea93fbe29c556`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:03:16.396509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-91311ad6-42fe-4b2f-adfb-a8d5939409dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:16.589977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 756/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Fri Sep 16 08:03:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:03:25.532773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b711c9b-378a-40bd-ace3-c70462c54c93 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:25.533311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b711c9b-378a-40bd-ace3-c70462c54c93 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:25.533311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b711c9b-378a-40bd-ace3-c70462c54c93 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:25.534694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 763/1519] 10.222.0.59 () {54 vars in 961 bytes} [Fri Sep 16 08:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:03:25.540143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ede43-8f10-47b1-8097-d1e5b0d48a4a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:25.540571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ede43-8f10-47b1-8097-d1e5b0d48a4a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:25.540730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ede43-8f10-47b1-8097-d1e5b0d48a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:25.560556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d56ede43-8f10-47b1-8097-d1e5b0d48a4a None None] MFA Rules not processed for user `c2dca05dfe8b4af0a02ea93fbe29c556`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:03:25.561765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d56ede43-8f10-47b1-8097-d1e5b0d48a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:25.570993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 757/1520] 10.222.0.59 () {58 vars in 1058 bytes} [Fri Sep 16 08:03:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:03:25.578622 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b711c9b-378a-40bd-ace3-c70462c54c93 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:03:25.592187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b711c9b-378a-40bd-ace3-c70462c54c93 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:03:25.597247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61a3e1-2315-491f-9520-c26812d8468f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:03:25.597843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a3e1-2315-491f-9520-c26812d8468f service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:25.598126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a3e1-2315-491f-9520-c26812d8468f service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:25.598398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a3e1-2315-491f-9520-c26812d8468f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:25.606596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61a3e1-2315-491f-9520-c26812d8468f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:03:25.608178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61a3e1-2315-491f-9520-c26812d8468f service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:03:25.618095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 764/1521] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:03:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:03:27.859663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907ae7d-fee6-49fa-aa4b-13143fef64ef None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:27.860144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907ae7d-fee6-49fa-aa4b-13143fef64ef None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:27.860252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907ae7d-fee6-49fa-aa4b-13143fef64ef None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:27.861599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 08:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:03:27.868478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffbfb6-8e7a-471c-b4fc-af0cdbacd7bf None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:27.868922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffbfb6-8e7a-471c-b4fc-af0cdbacd7bf None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:27.873896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffbfb6-8e7a-471c-b4fc-af0cdbacd7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:27.888652 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9fffbfb6-8e7a-471c-b4fc-af0cdbacd7bf None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:03:27.889874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9fffbfb6-8e7a-471c-b4fc-af0cdbacd7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:27.898466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 08:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:03:28.052877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907ae7d-fee6-49fa-aa4b-13143fef64ef None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:03:28.067315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907ae7d-fee6-49fa-aa4b-13143fef64ef None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:03:28.073391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce57fb-0dc1-469e-bc94-691844eb57a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:03:28.074021 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce57fb-0dc1-469e-bc94-691844eb57a8 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:28.074300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce57fb-0dc1-469e-bc94-691844eb57a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:28.074575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce57fb-0dc1-469e-bc94-691844eb57a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:28.082884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce57fb-0dc1-469e-bc94-691844eb57a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:03:28.085165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce57fb-0dc1-469e-bc94-691844eb57a8 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:03:28.094498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 759/1524] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:03:29.153565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b9dad-2634-4b38-80bf-6f7d15383b76 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:29.154092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b9dad-2634-4b38-80bf-6f7d15383b76 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:29.154092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b9dad-2634-4b38-80bf-6f7d15383b76 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:29.155465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 08:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:03:29.162273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2a4ef-1e43-489c-a38a-fdaf64e8934d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:29.162740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2a4ef-1e43-489c-a38a-fdaf64e8934d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:29.162854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2a4ef-1e43-489c-a38a-fdaf64e8934d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:29.181961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d0e2a4ef-1e43-489c-a38a-fdaf64e8934d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:03:29.183208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d0e2a4ef-1e43-489c-a38a-fdaf64e8934d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:29.190876 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 08:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:03:29.345794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b9dad-2634-4b38-80bf-6f7d15383b76 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:03:29.360786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b9dad-2634-4b38-80bf-6f7d15383b76 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:03:29.366082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6eb5ca-9d1b-4a52-a247-4ed089e7f0a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:03:29.366673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6eb5ca-9d1b-4a52-a247-4ed089e7f0a5 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:29.366975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6eb5ca-9d1b-4a52-a247-4ed089e7f0a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:29.367245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6eb5ca-9d1b-4a52-a247-4ed089e7f0a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:29.375064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6eb5ca-9d1b-4a52-a247-4ed089e7f0a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:03:29.376684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6eb5ca-9d1b-4a52-a247-4ed089e7f0a5 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:03:29.385781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 767/1527] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:03:33.258027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5b5ba-6206-41f3-bb05-d51e33924329 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:33.258643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5b5ba-6206-41f3-bb05-d51e33924329 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:33.258643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5b5ba-6206-41f3-bb05-d51e33924329 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:33.259926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 08:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:03:33.265021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf6e10-3f52-43e9-8055-1ca3a1aa8d07 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:33.265452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf6e10-3f52-43e9-8055-1ca3a1aa8d07 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:33.265562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf6e10-3f52-43e9-8055-1ca3a1aa8d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:33.285064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-91bf6e10-3f52-43e9-8055-1ca3a1aa8d07 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:03:33.286273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-91bf6e10-3f52-43e9-8055-1ca3a1aa8d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:33.294179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 08:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:03:33.583401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5b5ba-6206-41f3-bb05-d51e33924329 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:03:33.585098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cbb5b5ba-6206-41f3-bb05-d51e33924329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:33.627137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:03:33.642088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5b5ba-6206-41f3-bb05-d51e33924329 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:03:33.646083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f78f6-5ec7-41c7-bcf3-4c461edcc8da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:03:33.646708 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f78f6-5ec7-41c7-bcf3-4c461edcc8da service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:33.647014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f78f6-5ec7-41c7-bcf3-4c461edcc8da service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:33.647284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f78f6-5ec7-41c7-bcf3-4c461edcc8da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:33.655987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f78f6-5ec7-41c7-bcf3-4c461edcc8da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:03:33.658701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f78f6-5ec7-41c7-bcf3-4c461edcc8da service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:03:33.667559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 762/1530] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:03:35.780863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50490b50-3f2b-4500-8950-d5a5bfd1a273 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:35.781363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50490b50-3f2b-4500-8950-d5a5bfd1a273 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:35.781468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50490b50-3f2b-4500-8950-d5a5bfd1a273 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:35.782761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 08:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:03:35.788485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dc036-13f9-42a0-8400-18f02c18c02a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:35.788922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dc036-13f9-42a0-8400-18f02c18c02a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:35.789115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dc036-13f9-42a0-8400-18f02c18c02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:35.808792 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-068dc036-13f9-42a0-8400-18f02c18c02a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:03:35.809987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-068dc036-13f9-42a0-8400-18f02c18c02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:35.817725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 08:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:03:36.103317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50490b50-3f2b-4500-8950-d5a5bfd1a273 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:03:36.114396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50490b50-3f2b-4500-8950-d5a5bfd1a273 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:03:36.117953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd80ac-f4e1-409b-89a8-e6eb07201741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:03:36.118553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd80ac-f4e1-409b-89a8-e6eb07201741 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:36.118874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd80ac-f4e1-409b-89a8-e6eb07201741 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:36.119152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd80ac-f4e1-409b-89a8-e6eb07201741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:36.127804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd80ac-f4e1-409b-89a8-e6eb07201741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:03:36.129212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd80ac-f4e1-409b-89a8-e6eb07201741 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:03:36.139055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 770/1533] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:03:39.709989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729d8ef-8d00-41d0-a4d3-e228052c4a28 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:39.710565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729d8ef-8d00-41d0-a4d3-e228052c4a28 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:39.710565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729d8ef-8d00-41d0-a4d3-e228052c4a28 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:39.711877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 08:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:03:39.717212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70530a4d-c782-4ef9-944c-f76ebce31308 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:39.717600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70530a4d-c782-4ef9-944c-f76ebce31308 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:39.717788 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70530a4d-c782-4ef9-944c-f76ebce31308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:39.725856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=80 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:03:39.747334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-70530a4d-c782-4ef9-944c-f76ebce31308 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:03:39.747334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-70530a4d-c782-4ef9-944c-f76ebce31308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:39.748237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 08:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:03:40.049225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729d8ef-8d00-41d0-a4d3-e228052c4a28 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:03:40.050926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d729d8ef-8d00-41d0-a4d3-e228052c4a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:40.107171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729d8ef-8d00-41d0-a4d3-e228052c4a28 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:03:40.110667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb92bc-36a8-4200-921c-93a72c8513ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:03:40.111291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb92bc-36a8-4200-921c-93a72c8513ee service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:40.111572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb92bc-36a8-4200-921c-93a72c8513ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:40.111848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb92bc-36a8-4200-921c-93a72c8513ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:40.120858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb92bc-36a8-4200-921c-93a72c8513ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:03:40.122281 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb92bc-36a8-4200-921c-93a72c8513ee service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:03:40.131891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 765/1536] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:03:42.264860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea26a4-1a36-4486-aab0-e166dc75964c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:42.265455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea26a4-1a36-4486-aab0-e166dc75964c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:42.265455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea26a4-1a36-4486-aab0-e166dc75964c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:42.266782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 08:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:03:42.272045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26586b0b-0336-47f2-a8e2-7c5492b750ad None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:42.272458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26586b0b-0336-47f2-a8e2-7c5492b750ad None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:42.272569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26586b0b-0336-47f2-a8e2-7c5492b750ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:42.292361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-26586b0b-0336-47f2-a8e2-7c5492b750ad None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:03:42.293488 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-26586b0b-0336-47f2-a8e2-7c5492b750ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:42.301156 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 08:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:03:42.597118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea26a4-1a36-4486-aab0-e166dc75964c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:03:42.598811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cfea26a4-1a36-4486-aab0-e166dc75964c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:03:42.612370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea26a4-1a36-4486-aab0-e166dc75964c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:03:42.615890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc36d18-921c-48fb-b9df-8a01d31c0d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:03:42.616489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc36d18-921c-48fb-b9df-8a01d31c0d04 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:03:42.616762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc36d18-921c-48fb-b9df-8a01d31c0d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:03:42.617051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc36d18-921c-48fb-b9df-8a01d31c0d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:03:42.625648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc36d18-921c-48fb-b9df-8a01d31c0d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:03:42.627215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc36d18-921c-48fb-b9df-8a01d31c0d04 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:03:42.636281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 773/1539] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:03:52.286660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=41 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:03:52.632351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=8 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:05.981799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063813d0-d011-4081-9d57-a6f7f3d30ee9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:05.982457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063813d0-d011-4081-9d57-a6f7f3d30ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:05.982457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063813d0-d011-4081-9d57-a6f7f3d30ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.005462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-063813d0-d011-4081-9d57-a6f7f3d30ee9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.007692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-063813d0-d011-4081-9d57-a6f7f3d30ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.021864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 767/1540] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.031292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff57b5-31a0-4c69-a4b3-c84058d5d790 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.032452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff57b5-31a0-4c69-a4b3-c84058d5d790 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.033062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff57b5-31a0-4c69-a4b3-c84058d5d790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.059791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1aff57b5-31a0-4c69-a4b3-c84058d5d790 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.061518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1aff57b5-31a0-4c69-a4b3-c84058d5d790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.074308 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 774/1541] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.080184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063813d0-d011-4081-9d57-a6f7f3d30ee9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.091140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063813d0-d011-4081-9d57-a6f7f3d30ee9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.097091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f85726-ac70-4101-aa46-09bc1a262197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.097998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f85726-ac70-4101-aa46-09bc1a262197 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.098494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f85726-ac70-4101-aa46-09bc1a262197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.099094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f85726-ac70-4101-aa46-09bc1a262197 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.099967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f85726-ac70-4101-aa46-09bc1a262197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:06.100490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f85726-ac70-4101-aa46-09bc1a262197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.102945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f85726-ac70-4101-aa46-09bc1a262197 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.119115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 768/1542] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:06.127371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff57b5-31a0-4c69-a4b3-c84058d5d790 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.138534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff57b5-31a0-4c69-a4b3-c84058d5d790 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.146915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aec02-b2f2-4a37-a765-4c0d88a40be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.148449 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aec02-b2f2-4a37-a765-4c0d88a40be7 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.148947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aec02-b2f2-4a37-a765-4c0d88a40be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.149452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4aec02-b2f2-4a37-a765-4c0d88a40be7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.150325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4aec02-b2f2-4a37-a765-4c0d88a40be7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.155587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4aec02-b2f2-4a37-a765-4c0d88a40be7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.248136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30364a-476a-4af3-b8de-fb172dee195b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.248370 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30364a-476a-4af3-b8de-fb172dee195b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.248617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30364a-476a-4af3-b8de-fb172dee195b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.260397 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 775/1543] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/projects => generated 363 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:05:06.264169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aec02-b2f2-4a37-a765-4c0d88a40be7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.286416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dd30364a-476a-4af3-b8de-fb172dee195b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.288028 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dd30364a-476a-4af3-b8de-fb172dee195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.352283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4aec02-b2f2-4a37-a765-4c0d88a40be7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.358079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5f3c8-4640-42e4-84e9-d19a1a5e18f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.358704 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5f3c8-4640-42e4-84e9-d19a1a5e18f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.358986 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5f3c8-4640-42e4-84e9-d19a1a5e18f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.359255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5f3c8-4640-42e4-84e9-d19a1a5e18f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.359908 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5f3c8-4640-42e4-84e9-d19a1a5e18f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.360914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-5be5f3c8-4640-42e4-84e9-d19a1a5e18f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:05:06.360914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 16 08:05:06.360914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: Sep 16 08:05:06.362115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5f3c8-4640-42e4-84e9-d19a1a5e18f4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.363572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 769/1544] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:06.364814 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14725b6f-adbd-46b0-9f2b-47a327391e34 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.365099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14725b6f-adbd-46b0-9f2b-47a327391e34 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.365371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14725b6f-adbd-46b0-9f2b-47a327391e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.389408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-14725b6f-adbd-46b0-9f2b-47a327391e34 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.390616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-14725b6f-adbd-46b0-9f2b-47a327391e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.398410 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 770/1545] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.399751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbc97b-dc0b-4e47-86f1-4f8c6defb80e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.400036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbc97b-dc0b-4e47-86f1-4f8c6defb80e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.400312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbc97b-dc0b-4e47-86f1-4f8c6defb80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.419136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9cfbc97b-dc0b-4e47-86f1-4f8c6defb80e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.421354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9cfbc97b-dc0b-4e47-86f1-4f8c6defb80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.427983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 771/1546] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.429752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751ee59-12f9-41ee-b43a-2b2620b97cb9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.430037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751ee59-12f9-41ee-b43a-2b2620b97cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.430318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751ee59-12f9-41ee-b43a-2b2620b97cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.450358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1751ee59-12f9-41ee-b43a-2b2620b97cb9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.451579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1751ee59-12f9-41ee-b43a-2b2620b97cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.459230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 772/1547] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.466349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e60796-fb4d-471e-bab4-bb026ab762d8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.466349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e60796-fb4d-471e-bab4-bb026ab762d8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.466349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e60796-fb4d-471e-bab4-bb026ab762d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.472862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 776/1548] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/users => generated 282 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:06.472862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec30644-0251-4c04-aee0-2cbf2ac23190 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.472862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec30644-0251-4c04-aee0-2cbf2ac23190 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.472862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec30644-0251-4c04-aee0-2cbf2ac23190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.480397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c9e60796-fb4d-471e-bab4-bb026ab762d8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.481585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c9e60796-fb4d-471e-bab4-bb026ab762d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.488903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2ec30644-0251-4c04-aee0-2cbf2ac23190 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.489464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 773/1549] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.490069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2ec30644-0251-4c04-aee0-2cbf2ac23190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.490739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb15bfa-54fc-4268-8612-39a73c3b6283 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.491027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb15bfa-54fc-4268-8612-39a73c3b6283 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.491300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb15bfa-54fc-4268-8612-39a73c3b6283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.498963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 777/1550] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.500100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b999e-6ab4-476c-8496-6f6df0ae72e5 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.500390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b999e-6ab4-476c-8496-6f6df0ae72e5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.500662 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b999e-6ab4-476c-8496-6f6df0ae72e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.512141 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1fb15bfa-54fc-4268-8612-39a73c3b6283 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.513339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1fb15bfa-54fc-4268-8612-39a73c3b6283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.520486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-022b999e-6ab4-476c-8496-6f6df0ae72e5 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.521218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 774/1551] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.521736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-022b999e-6ab4-476c-8496-6f6df0ae72e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.522462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65015c-3cc7-49fe-88a5-00f313c81087 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.522813 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65015c-3cc7-49fe-88a5-00f313c81087 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.523088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65015c-3cc7-49fe-88a5-00f313c81087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.532718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 778/1552] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.532718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b64b2-07a8-4503-b696-ad493bec37f8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.532718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b64b2-07a8-4503-b696-ad493bec37f8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.533058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b64b2-07a8-4503-b696-ad493bec37f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.543113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0e65015c-3cc7-49fe-88a5-00f313c81087 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.544316 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0e65015c-3cc7-49fe-88a5-00f313c81087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.552516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 775/1553] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.552786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5c2b64b2-07a8-4503-b696-ad493bec37f8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.553764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570674ff-fe52-416f-97bb-b31e4fefe58c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.553954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5c2b64b2-07a8-4503-b696-ad493bec37f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.554582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570674ff-fe52-416f-97bb-b31e4fefe58c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.554871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570674ff-fe52-416f-97bb-b31e4fefe58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.562323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 779/1554] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.563850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b64b2-07a8-4503-b696-ad493bec37f8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.573769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b64b2-07a8-4503-b696-ad493bec37f8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.574985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-570674ff-fe52-416f-97bb-b31e4fefe58c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.576178 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-570674ff-fe52-416f-97bb-b31e4fefe58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.578367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da327218-a820-4a2b-b352-5b8267e211fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.578976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da327218-a820-4a2b-b352-5b8267e211fd admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.579254 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da327218-a820-4a2b-b352-5b8267e211fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.579523 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da327218-a820-4a2b-b352-5b8267e211fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.580156 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da327218-a820-4a2b-b352-5b8267e211fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.581731 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da327218-a820-4a2b-b352-5b8267e211fd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.583906 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 776/1555] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.585229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13e4b-969e-48dd-9a8d-db073e9876ff None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.585604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13e4b-969e-48dd-9a8d-db073e9876ff None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.585879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13e4b-969e-48dd-9a8d-db073e9876ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.595164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 780/1556] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:06.595164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb5e35-9636-4b15-8003-94dd4209443a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.595164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb5e35-9636-4b15-8003-94dd4209443a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.595164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb5e35-9636-4b15-8003-94dd4209443a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.605414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c9b13e4b-969e-48dd-9a8d-db073e9876ff None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.606605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c9b13e4b-969e-48dd-9a8d-db073e9876ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.612757 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-15cb5e35-9636-4b15-8003-94dd4209443a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.613927 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-15cb5e35-9636-4b15-8003-94dd4209443a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.614354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 777/1557] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.615613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c553c-dbd2-48a3-878c-b028dc196063 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.615900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c553c-dbd2-48a3-878c-b028dc196063 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.616173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c553c-dbd2-48a3-878c-b028dc196063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.623814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 781/1558] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.624595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb5e35-9636-4b15-8003-94dd4209443a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.634371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb5e35-9636-4b15-8003-94dd4209443a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.638908 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea3727-6a86-4d49-8371-a141bd8cd947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.639497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea3727-6a86-4d49-8371-a141bd8cd947 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.639602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7a7c553c-dbd2-48a3-878c-b028dc196063 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:06.639778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea3727-6a86-4d49-8371-a141bd8cd947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.640046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea3727-6a86-4d49-8371-a141bd8cd947 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.640711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea3727-6a86-4d49-8371-a141bd8cd947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:06.640830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7a7c553c-dbd2-48a3-878c-b028dc196063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:06.641010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea3727-6a86-4d49-8371-a141bd8cd947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.642010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-4fea3727-6a86-4d49-8371-a141bd8cd947 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:05:06.642010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 16 08:05:06.642010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: Sep 16 08:05:06.642911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea3727-6a86-4d49-8371-a141bd8cd947 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.649103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 778/1559] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:06.653378 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c553c-dbd2-48a3-878c-b028dc196063 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.654568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 782/1560] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:06.655392 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea3727-6a86-4d49-8371-a141bd8cd947 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.663663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c553c-dbd2-48a3-878c-b028dc196063 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.664830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea3727-6a86-4d49-8371-a141bd8cd947 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.668250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0da0f-b8e0-41ae-bec1-2fba745d396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.668867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0da0f-b8e0-41ae-bec1-2fba745d396d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.669161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0da0f-b8e0-41ae-bec1-2fba745d396d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.669283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a710dcb-7ca3-4a1e-837b-e530d3c50048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.669400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0da0f-b8e0-41ae-bec1-2fba745d396d admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.669805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a710dcb-7ca3-4a1e-837b-e530d3c50048 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.670034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0da0f-b8e0-41ae-bec1-2fba745d396d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:06.670156 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a710dcb-7ca3-4a1e-837b-e530d3c50048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.670335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0da0f-b8e0-41ae-bec1-2fba745d396d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.670443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a710dcb-7ca3-4a1e-837b-e530d3c50048 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.670996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a710dcb-7ca3-4a1e-837b-e530d3c50048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:06.671298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a710dcb-7ca3-4a1e-837b-e530d3c50048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.671957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0da0f-b8e0-41ae-bec1-2fba745d396d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.672855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a710dcb-7ca3-4a1e-837b-e530d3c50048 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.683928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 779/1561] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:06.684748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0da0f-b8e0-41ae-bec1-2fba745d396d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.692494 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 783/1562] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:06.692494 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a710dcb-7ca3-4a1e-837b-e530d3c50048 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.702053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0da0f-b8e0-41ae-bec1-2fba745d396d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.703330 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a710dcb-7ca3-4a1e-837b-e530d3c50048 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.706633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388172d-a343-4738-8a44-0d5689e78344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.707265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388172d-a343-4738-8a44-0d5689e78344 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.707541 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388172d-a343-4738-8a44-0d5689e78344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.707802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8688893-62f5-4268-a10d-5b028a4c4017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.707921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388172d-a343-4738-8a44-0d5689e78344 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.708396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8688893-62f5-4268-a10d-5b028a4c4017 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.708506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388172d-a343-4738-8a44-0d5689e78344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:06.708675 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8688893-62f5-4268-a10d-5b028a4c4017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.708771 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388172d-a343-4738-8a44-0d5689e78344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.708947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8688893-62f5-4268-a10d-5b028a4c4017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.709580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8688893-62f5-4268-a10d-5b028a4c4017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.710371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388172d-a343-4738-8a44-0d5689e78344 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.711231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8688893-62f5-4268-a10d-5b028a4c4017 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.721163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 784/1563] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:06.721163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8688893-62f5-4268-a10d-5b028a4c4017 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.726166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 780/1564] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:06.727015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388172d-a343-4738-8a44-0d5689e78344 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.733403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8688893-62f5-4268-a10d-5b028a4c4017 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.738142 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4a9c7-eca4-4984-af2b-0d4c9e7005dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.740025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4a9c7-eca4-4984-af2b-0d4c9e7005dc admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.740025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4a9c7-eca4-4984-af2b-0d4c9e7005dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.740025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4a9c7-eca4-4984-af2b-0d4c9e7005dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.740025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4a9c7-eca4-4984-af2b-0d4c9e7005dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:06.740385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4a9c7-eca4-4984-af2b-0d4c9e7005dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.742920 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4a9c7-eca4-4984-af2b-0d4c9e7005dc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.745409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388172d-a343-4738-8a44-0d5689e78344 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.749989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373eb9b-d82c-4a6e-baef-531647c0aa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.750585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373eb9b-d82c-4a6e-baef-531647c0aa89 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.750895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373eb9b-d82c-4a6e-baef-531647c0aa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.751171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373eb9b-d82c-4a6e-baef-531647c0aa89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.751812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373eb9b-d82c-4a6e-baef-531647c0aa89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:06.752116 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373eb9b-d82c-4a6e-baef-531647c0aa89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.753737 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373eb9b-d82c-4a6e-baef-531647c0aa89 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.756277 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 785/1565] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:06.757080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4a9c7-eca4-4984-af2b-0d4c9e7005dc admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.767424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4a9c7-eca4-4984-af2b-0d4c9e7005dc admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.777203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 781/1566] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:06.777368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e0351-8dc8-4131-892c-f8f82383d056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.777368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e0351-8dc8-4131-892c-f8f82383d056 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.777559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373eb9b-d82c-4a6e-baef-531647c0aa89 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.777658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e0351-8dc8-4131-892c-f8f82383d056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.777658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e0351-8dc8-4131-892c-f8f82383d056 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.777658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e0351-8dc8-4131-892c-f8f82383d056 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:06.777658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e0351-8dc8-4131-892c-f8f82383d056 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.777658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e0351-8dc8-4131-892c-f8f82383d056 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.782263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373eb9b-d82c-4a6e-baef-531647c0aa89 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.787057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebdbce-b207-4246-8c5c-dc6746c12980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.787758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebdbce-b207-4246-8c5c-dc6746c12980 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.787933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebdbce-b207-4246-8c5c-dc6746c12980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.788202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebdbce-b207-4246-8c5c-dc6746c12980 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.789337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebdbce-b207-4246-8c5c-dc6746c12980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.790177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 786/1567] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:06.790340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-67ebdbce-b207-4246-8c5c-dc6746c12980 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:05:06.790340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 16 08:05:06.790340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: Sep 16 08:05:06.790827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e0351-8dc8-4131-892c-f8f82383d056 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.791552 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebdbce-b207-4246-8c5c-dc6746c12980 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.801697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e0351-8dc8-4131-892c-f8f82383d056 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.806811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8ed14-d624-42a8-a8fa-56dfd7e4b128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:06.807395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8ed14-d624-42a8-a8fa-56dfd7e4b128 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:06.807885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8ed14-d624-42a8-a8fa-56dfd7e4b128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:06.808024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8ed14-d624-42a8-a8fa-56dfd7e4b128 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:06.808545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8ed14-d624-42a8-a8fa-56dfd7e4b128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:06.810430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8ed14-d624-42a8-a8fa-56dfd7e4b128 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:06.901455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 782/1568] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/projects => generated 405 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:06.902282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebdbce-b207-4246-8c5c-dc6746c12980 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.923085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:06.930532 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:06.941028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 787/1569] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:06.941028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8ed14-d624-42a8-a8fa-56dfd7e4b128 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:06.964718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebdbce-b207-4246-8c5c-dc6746c12980 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:06.999726 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8ed14-d624-42a8-a8fa-56dfd7e4b128 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:07.005147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d028ce5-4a24-4a7d-8a99-c8b283416edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.005757 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d028ce5-4a24-4a7d-8a99-c8b283416edb admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.006038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d028ce5-4a24-4a7d-8a99-c8b283416edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.006319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d028ce5-4a24-4a7d-8a99-c8b283416edb admin admin] PATH_INFO: `/v3/projects/818276d7e2d74ad6b1d24b201f75acb2/users/b5895120931b431884b32ce8ffb704af/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.020367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b3bac-4fd7-4b30-b754-b94e0fed768d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.020942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d028ce5-4a24-4a7d-8a99-c8b283416edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=818276d7e2d74ad6b1d24b201f75acb2, user_id=b5895120931b431884b32ce8ffb704af, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.021129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b3bac-4fd7-4b30-b754-b94e0fed768d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.021475 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b3bac-4fd7-4b30-b754-b94e0fed768d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.021837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b3bac-4fd7-4b30-b754-b94e0fed768d admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.022517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b3bac-4fd7-4b30-b754-b94e0fed768d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.023496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d028ce5-4a24-4a7d-8a99-c8b283416edb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:07.024523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b3bac-4fd7-4b30-b754-b94e0fed768d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:07.112070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 788/1570] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:06 2022] PUT /identity/v3/projects/818276d7e2d74ad6b1d24b201f75acb2/users/b5895120931b431884b32ce8ffb704af/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:07.112897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d028ce5-4a24-4a7d-8a99-c8b283416edb admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:07.126818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 783/1571] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:06 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:07.126818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b3bac-4fd7-4b30-b754-b94e0fed768d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:07.183215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d028ce5-4a24-4a7d-8a99-c8b283416edb admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:07.186610 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b3bac-4fd7-4b30-b754-b94e0fed768d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:07.187755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc32f9-8c7b-4ed3-bec8-e11ec35bc5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.188360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc32f9-8c7b-4ed3-bec8-e11ec35bc5e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.188814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc32f9-8c7b-4ed3-bec8-e11ec35bc5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.189080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc32f9-8c7b-4ed3-bec8-e11ec35bc5e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.189703 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc32f9-8c7b-4ed3-bec8-e11ec35bc5e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.191637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc32f9-8c7b-4ed3-bec8-e11ec35bc5e6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:07.191785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a434f-83a4-4b53-9697-d2e12437e4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.192259 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a434f-83a4-4b53-9697-d2e12437e4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.192542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a434f-83a4-4b53-9697-d2e12437e4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.192819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a434f-83a4-4b53-9697-d2e12437e4f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.193445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a434f-83a4-4b53-9697-d2e12437e4f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.195403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a434f-83a4-4b53-9697-d2e12437e4f5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:07.324833 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 784/1572] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:07 2022] POST /identity/v3/projects => generated 365 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:07.325569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a434f-83a4-4b53-9697-d2e12437e4f5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:07.337150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 789/1573] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:07 2022] POST /identity/v3/projects => generated 377 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:07.338441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc32f9-8c7b-4ed3-bec8-e11ec35bc5e6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:07.367450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:07.387457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a434f-83a4-4b53-9697-d2e12437e4f5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:07.403730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc32f9-8c7b-4ed3-bec8-e11ec35bc5e6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:07.408751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83ae57-f9a9-4008-8faa-65b6caa095e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.410511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83ae57-f9a9-4008-8faa-65b6caa095e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.410511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83ae57-f9a9-4008-8faa-65b6caa095e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.410511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83ae57-f9a9-4008-8faa-65b6caa095e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.410511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83ae57-f9a9-4008-8faa-65b6caa095e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.412686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83ae57-f9a9-4008-8faa-65b6caa095e7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:07.425291 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:07.521689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2969b1b-963a-4dcd-b363-926a27429562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.523387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2969b1b-963a-4dcd-b363-926a27429562 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.523387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2969b1b-963a-4dcd-b363-926a27429562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.523606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2969b1b-963a-4dcd-b363-926a27429562 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.524147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2969b1b-963a-4dcd-b363-926a27429562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.529458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2969b1b-963a-4dcd-b363-926a27429562 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:07.529552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 790/1574] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:07 2022] POST /identity/v3/projects => generated 391 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:07.529552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83ae57-f9a9-4008-8faa-65b6caa095e7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:07.590015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83ae57-f9a9-4008-8faa-65b6caa095e7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:07.594579 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043daafe-42a8-4d32-b3de-178358b57158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.595212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043daafe-42a8-4d32-b3de-178358b57158 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.595495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043daafe-42a8-4d32-b3de-178358b57158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.595794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043daafe-42a8-4d32-b3de-178358b57158 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.596396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043daafe-42a8-4d32-b3de-178358b57158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.598357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043daafe-42a8-4d32-b3de-178358b57158 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:07.656381 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 785/1575] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:07 2022] POST /identity/v3/projects => generated 377 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:07.657193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2969b1b-963a-4dcd-b363-926a27429562 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:07.686096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 791/1576] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:07 2022] POST /identity/v3/users => generated 303 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:07.686096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec41477-f53c-4e4c-b1eb-f0893e821bca None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.686096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec41477-f53c-4e4c-b1eb-f0893e821bca None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.686096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec41477-f53c-4e4c-b1eb-f0893e821bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.716292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1ec41477-f53c-4e4c-b1eb-f0893e821bca None None] MFA Rules not processed for user `b5895120931b431884b32ce8ffb704af`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:07.717524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1ec41477-f53c-4e4c-b1eb-f0893e821bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:07.722444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2969b1b-963a-4dcd-b363-926a27429562 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:07.725694 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe916dc1-f1e4-484a-9597-6c1c908eb396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.726312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe916dc1-f1e4-484a-9597-6c1c908eb396 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.726591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe916dc1-f1e4-484a-9597-6c1c908eb396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.726898 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe916dc1-f1e4-484a-9597-6c1c908eb396 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.727557 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe916dc1-f1e4-484a-9597-6c1c908eb396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.729494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe916dc1-f1e4-484a-9597-6c1c908eb396 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:07.784076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 792/1577] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:07.784905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec41477-f53c-4e4c-b1eb-f0893e821bca None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:07.796619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec41477-f53c-4e4c-b1eb-f0893e821bca None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:07.800991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ec38a-eac3-4ae9-9d39-fc2f8643da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.801563 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ec38a-eac3-4ae9-9d39-fc2f8643da67 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.801838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ec38a-eac3-4ae9-9d39-fc2f8643da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.802105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ec38a-eac3-4ae9-9d39-fc2f8643da67 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.802744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ec38a-eac3-4ae9-9d39-fc2f8643da67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.804633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ec38a-eac3-4ae9-9d39-fc2f8643da67 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:07.815891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 786/1578] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:07 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:07.815891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe916dc1-f1e4-484a-9597-6c1c908eb396 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:07.825053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe916dc1-f1e4-484a-9597-6c1c908eb396 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:07.829712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada368-f000-4c65-aa8f-b5dfb206fa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.830305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada368-f000-4c65-aa8f-b5dfb206fa80 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.830583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada368-f000-4c65-aa8f-b5dfb206fa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.830895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada368-f000-4c65-aa8f-b5dfb206fa80 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.831516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cada368-f000-4c65-aa8f-b5dfb206fa80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.833541 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cada368-f000-4c65-aa8f-b5dfb206fa80 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:07.904697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 793/1579] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:07 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:07.905469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ec38a-eac3-4ae9-9d39-fc2f8643da67 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:07.915209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ec38a-eac3-4ae9-9d39-fc2f8643da67 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:07.919772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821b564-fd5c-4917-9a66-92062be803ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.920335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821b564-fd5c-4917-9a66-92062be803ea admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.920615 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821b564-fd5c-4917-9a66-92062be803ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.920887 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821b564-fd5c-4917-9a66-92062be803ea admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.921495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821b564-fd5c-4917-9a66-92062be803ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.923429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821b564-fd5c-4917-9a66-92062be803ea admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:07.964620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 787/1580] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:07 2022] POST /identity/v3/users => generated 283 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:07.964620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada368-f000-4c65-aa8f-b5dfb206fa80 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:07.971929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada368-f000-4c65-aa8f-b5dfb206fa80 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:07.976478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64f082-af7d-4cc7-b0e5-890adcbc7aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:07.977114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64f082-af7d-4cc7-b0e5-890adcbc7aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:07.977399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64f082-af7d-4cc7-b0e5-890adcbc7aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:07.977676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64f082-af7d-4cc7-b0e5-890adcbc7aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:07.978294 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64f082-af7d-4cc7-b0e5-890adcbc7aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:07.980262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64f082-af7d-4cc7-b0e5-890adcbc7aa0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.021740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 794/1581] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:07 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:08.021740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821b564-fd5c-4917-9a66-92062be803ea admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.029111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821b564-fd5c-4917-9a66-92062be803ea admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.033583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1162c1-be39-4afe-8b87-13c362260110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.034243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1162c1-be39-4afe-8b87-13c362260110 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.034518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1162c1-be39-4afe-8b87-13c362260110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.034822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1162c1-be39-4afe-8b87-13c362260110 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.035431 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1162c1-be39-4afe-8b87-13c362260110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.037605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1162c1-be39-4afe-8b87-13c362260110 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.070844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 788/1582] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:07 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:08.071681 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64f082-af7d-4cc7-b0e5-890adcbc7aa0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.081024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64f082-af7d-4cc7-b0e5-890adcbc7aa0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.085488 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ae121-9816-4531-858f-475c0db92615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.086051 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ae121-9816-4531-858f-475c0db92615 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.086326 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ae121-9816-4531-858f-475c0db92615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.086596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ae121-9816-4531-858f-475c0db92615 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.087251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ae121-9816-4531-858f-475c0db92615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.088823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ae121-9816-4531-858f-475c0db92615 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.098477 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 789/1583] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.099241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ae121-9816-4531-858f-475c0db92615 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.108244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ae121-9816-4531-858f-475c0db92615 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.112750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d956193-5a49-4299-95c0-16e6d6d55017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.113282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d956193-5a49-4299-95c0-16e6d6d55017 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.113562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d956193-5a49-4299-95c0-16e6d6d55017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.113864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d956193-5a49-4299-95c0-16e6d6d55017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.114469 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d956193-5a49-4299-95c0-16e6d6d55017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.116327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d956193-5a49-4299-95c0-16e6d6d55017 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.122002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 795/1584] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:08 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:08.122702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1162c1-be39-4afe-8b87-13c362260110 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.129498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:08.133888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1162c1-be39-4afe-8b87-13c362260110 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.138853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50484833-e600-4a0f-adca-0f57b3d58afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.139441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50484833-e600-4a0f-adca-0f57b3d58afe admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.139716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50484833-e600-4a0f-adca-0f57b3d58afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.139985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50484833-e600-4a0f-adca-0f57b3d58afe admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.140646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50484833-e600-4a0f-adca-0f57b3d58afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.142276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50484833-e600-4a0f-adca-0f57b3d58afe admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.152208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 796/1585] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.152980 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50484833-e600-4a0f-adca-0f57b3d58afe admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.162558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50484833-e600-4a0f-adca-0f57b3d58afe admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.166994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a381777-15f1-43eb-b5b2-3f1dd1eda784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.167538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a381777-15f1-43eb-b5b2-3f1dd1eda784 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.167815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a381777-15f1-43eb-b5b2-3f1dd1eda784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.168190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a381777-15f1-43eb-b5b2-3f1dd1eda784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.168824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a381777-15f1-43eb-b5b2-3f1dd1eda784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.170325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a381777-15f1-43eb-b5b2-3f1dd1eda784 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.171562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:08.180749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 797/1586] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.181529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a381777-15f1-43eb-b5b2-3f1dd1eda784 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.191644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a381777-15f1-43eb-b5b2-3f1dd1eda784 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.196840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458d2e4-8156-48c5-a5a4-dc5c78e66f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.197429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458d2e4-8156-48c5-a5a4-dc5c78e66f66 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.197708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458d2e4-8156-48c5-a5a4-dc5c78e66f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.197978 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458d2e4-8156-48c5-a5a4-dc5c78e66f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.198612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458d2e4-8156-48c5-a5a4-dc5c78e66f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.200201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458d2e4-8156-48c5-a5a4-dc5c78e66f66 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.210811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 790/1587] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:08 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:08.211147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 798/1588] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.211993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d956193-5a49-4299-95c0-16e6d6d55017 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.213706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458d2e4-8156-48c5-a5a4-dc5c78e66f66 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.223124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458d2e4-8156-48c5-a5a4-dc5c78e66f66 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.276486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d956193-5a49-4299-95c0-16e6d6d55017 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.281572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f598b60-ed95-4ed5-9812-4c8bde9a032f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.282068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012102f-edfb-49ab-b108-023c8a9dcf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.282228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f598b60-ed95-4ed5-9812-4c8bde9a032f admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.282515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f598b60-ed95-4ed5-9812-4c8bde9a032f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.282687 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012102f-edfb-49ab-b108-023c8a9dcf41 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.282828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f598b60-ed95-4ed5-9812-4c8bde9a032f admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.282983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012102f-edfb-49ab-b108-023c8a9dcf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.283255 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012102f-edfb-49ab-b108-023c8a9dcf41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.283629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f598b60-ed95-4ed5-9812-4c8bde9a032f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.283893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012102f-edfb-49ab-b108-023c8a9dcf41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.285227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f598b60-ed95-4ed5-9812-4c8bde9a032f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.285531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012102f-edfb-49ab-b108-023c8a9dcf41 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.299712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 799/1589] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.299952 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 791/1590] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.300058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f598b60-ed95-4ed5-9812-4c8bde9a032f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.300157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012102f-edfb-49ab-b108-023c8a9dcf41 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.308046 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012102f-edfb-49ab-b108-023c8a9dcf41 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.309111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f598b60-ed95-4ed5-9812-4c8bde9a032f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.312555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6453e07-6ec8-4dd1-88ec-232d29994115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.313142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453e07-6ec8-4dd1-88ec-232d29994115 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.313420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453e07-6ec8-4dd1-88ec-232d29994115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.313693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453e07-6ec8-4dd1-88ec-232d29994115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.313988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba73da-4c78-4f42-ab62-b6919b798461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.314319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6453e07-6ec8-4dd1-88ec-232d29994115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.314611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba73da-4c78-4f42-ab62-b6919b798461 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.314959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba73da-4c78-4f42-ab62-b6919b798461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.315238 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba73da-4c78-4f42-ab62-b6919b798461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.315881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba73da-4c78-4f42-ab62-b6919b798461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.316011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6453e07-6ec8-4dd1-88ec-232d29994115 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.317489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba73da-4c78-4f42-ab62-b6919b798461 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.325914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 792/1591] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.328917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6453e07-6ec8-4dd1-88ec-232d29994115 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.329038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 800/1592] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.329038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba73da-4c78-4f42-ab62-b6919b798461 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.337730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6453e07-6ec8-4dd1-88ec-232d29994115 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.338557 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba73da-4c78-4f42-ab62-b6919b798461 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.342886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc198df-554e-42b0-96d0-7836239383e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.343283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758fe8d-eaad-4cb0-ac4b-c12fe65d35b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.343467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc198df-554e-42b0-96d0-7836239383e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.343746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc198df-554e-42b0-96d0-7836239383e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.343853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758fe8d-eaad-4cb0-ac4b-c12fe65d35b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.344019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc198df-554e-42b0-96d0-7836239383e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.344124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758fe8d-eaad-4cb0-ac4b-c12fe65d35b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.344396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758fe8d-eaad-4cb0-ac4b-c12fe65d35b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.344710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc198df-554e-42b0-96d0-7836239383e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.345090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758fe8d-eaad-4cb0-ac4b-c12fe65d35b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.346247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc198df-554e-42b0-96d0-7836239383e8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.346730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758fe8d-eaad-4cb0-ac4b-c12fe65d35b1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.357349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 801/1594] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.357516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 793/1594] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.361209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc198df-554e-42b0-96d0-7836239383e8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.361350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758fe8d-eaad-4cb0-ac4b-c12fe65d35b1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.371168 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc198df-554e-42b0-96d0-7836239383e8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.371526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758fe8d-eaad-4cb0-ac4b-c12fe65d35b1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.375650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193abb5-e947-4c81-92f4-565c44c6093b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.376224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193abb5-e947-4c81-92f4-565c44c6093b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.376495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab9795-9cd6-4109-9c67-c731a15285cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.376615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193abb5-e947-4c81-92f4-565c44c6093b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.376763 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193abb5-e947-4c81-92f4-565c44c6093b admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.377253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab9795-9cd6-4109-9c67-c731a15285cd admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.377380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193abb5-e947-4c81-92f4-565c44c6093b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.377596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab9795-9cd6-4109-9c67-c731a15285cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.377871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab9795-9cd6-4109-9c67-c731a15285cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.378505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab9795-9cd6-4109-9c67-c731a15285cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.379318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193abb5-e947-4c81-92f4-565c44c6093b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.380130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab9795-9cd6-4109-9c67-c731a15285cd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.394017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 802/1595] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.394017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab9795-9cd6-4109-9c67-c731a15285cd admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.401003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab9795-9cd6-4109-9c67-c731a15285cd admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.405507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345bfe2-0897-4429-a117-fec636835af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.408681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345bfe2-0897-4429-a117-fec636835af9 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.408681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345bfe2-0897-4429-a117-fec636835af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.408681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345bfe2-0897-4429-a117-fec636835af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.408681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345bfe2-0897-4429-a117-fec636835af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.409024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345bfe2-0897-4429-a117-fec636835af9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.419374 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 803/1596] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.420139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345bfe2-0897-4429-a117-fec636835af9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.429762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345bfe2-0897-4429-a117-fec636835af9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.434321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a1bac-6c5c-4ea2-9f30-0bd21b2d773b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.434923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1bac-6c5c-4ea2-9f30-0bd21b2d773b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.435198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1bac-6c5c-4ea2-9f30-0bd21b2d773b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.435469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1bac-6c5c-4ea2-9f30-0bd21b2d773b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.436068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a1bac-6c5c-4ea2-9f30-0bd21b2d773b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.437676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a1bac-6c5c-4ea2-9f30-0bd21b2d773b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.447193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 804/1597] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.447990 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a1bac-6c5c-4ea2-9f30-0bd21b2d773b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.456864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a1bac-6c5c-4ea2-9f30-0bd21b2d773b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.461391 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e36417-8625-4779-88bb-ab829d0bba08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.461943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e36417-8625-4779-88bb-ab829d0bba08 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.462223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e36417-8625-4779-88bb-ab829d0bba08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.462493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e36417-8625-4779-88bb-ab829d0bba08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.463131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e36417-8625-4779-88bb-ab829d0bba08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.464849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e36417-8625-4779-88bb-ab829d0bba08 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 794/1598] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:08 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:08.465695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193abb5-e947-4c81-92f4-565c44c6093b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.475399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193abb5-e947-4c81-92f4-565c44c6093b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.479765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 805/1599] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:08.479765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e36417-8625-4779-88bb-ab829d0bba08 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.481052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b796842-9e7e-458a-b1e8-07d48beb8685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.481716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b796842-9e7e-458a-b1e8-07d48beb8685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.481998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b796842-9e7e-458a-b1e8-07d48beb8685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.482272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b796842-9e7e-458a-b1e8-07d48beb8685 admin admin] PATH_INFO: `/v3/projects/8f789d3ecccd4440a05d7bfd944c98bd/users/47aec2c802084ca6b949b757692997bc/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.486786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e36417-8625-4779-88bb-ab829d0bba08 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.491238 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfa7f2-33e9-4bca-81f4-8bb0f83eb5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.492414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfa7f2-33e9-4bca-81f4-8bb0f83eb5a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.492690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfa7f2-33e9-4bca-81f4-8bb0f83eb5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.492963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfa7f2-33e9-4bca-81f4-8bb0f83eb5a2 admin admin] PATH_INFO: `/v3/projects/5e900d236439454284220179da162c08/users/5e123f7323f3463fa812e51edf28d2ee/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.498483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b796842-9e7e-458a-b1e8-07d48beb8685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f789d3ecccd4440a05d7bfd944c98bd, user_id=47aec2c802084ca6b949b757692997bc, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.499569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-9b796842-9e7e-458a-b1e8-07d48beb8685 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:05:08.499569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 16 08:05:08.499569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: Sep 16 08:05:08.501535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b796842-9e7e-458a-b1e8-07d48beb8685 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.507080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfa7f2-33e9-4bca-81f4-8bb0f83eb5a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e900d236439454284220179da162c08, user_id=5e123f7323f3463fa812e51edf28d2ee, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.509681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfa7f2-33e9-4bca-81f4-8bb0f83eb5a2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.590108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 795/1600] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:08 2022] PUT /identity/v3/projects/8f789d3ecccd4440a05d7bfd944c98bd/users/47aec2c802084ca6b949b757692997bc/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:08.590933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b796842-9e7e-458a-b1e8-07d48beb8685 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.603226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 806/1601] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:08 2022] PUT /identity/v3/projects/5e900d236439454284220179da162c08/users/5e123f7323f3463fa812e51edf28d2ee/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:08.604278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfa7f2-33e9-4bca-81f4-8bb0f83eb5a2 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.654996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b796842-9e7e-458a-b1e8-07d48beb8685 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.655754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:08.667724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfa7f2-33e9-4bca-81f4-8bb0f83eb5a2 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.672588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d19e1-1ec4-4d4a-b027-0ea52b4a9a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.673254 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d19e1-1ec4-4d4a-b027-0ea52b4a9a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.673539 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d19e1-1ec4-4d4a-b027-0ea52b4a9a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.673810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d19e1-1ec4-4d4a-b027-0ea52b4a9a61 admin admin] PATH_INFO: `/v3/projects/ac6baea7843b47109de92534d1934eed/users/755487a8ea3d48a58983efea5c7a1d62/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.685288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:08.688351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d19e1-1ec4-4d4a-b027-0ea52b4a9a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6baea7843b47109de92534d1934eed, user_id=755487a8ea3d48a58983efea5c7a1d62, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.691570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d19e1-1ec4-4d4a-b027-0ea52b4a9a61 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.713131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e94889-d990-4c48-9ced-f1d3673cd153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.713794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e94889-d990-4c48-9ced-f1d3673cd153 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.714073 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e94889-d990-4c48-9ced-f1d3673cd153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.714348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e94889-d990-4c48-9ced-f1d3673cd153 admin admin] PATH_INFO: `/v3/projects/b807736ef6c54be2b4715f32a4cfea57/users/ea2f6b804e164dbcb4f67926f23fdd59/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.729750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e94889-d990-4c48-9ced-f1d3673cd153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b807736ef6c54be2b4715f32a4cfea57, user_id=ea2f6b804e164dbcb4f67926f23fdd59, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.732447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e94889-d990-4c48-9ced-f1d3673cd153 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.769580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 807/1602] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:08 2022] PUT /identity/v3/projects/ac6baea7843b47109de92534d1934eed/users/755487a8ea3d48a58983efea5c7a1d62/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:08.771136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d19e1-1ec4-4d4a-b027-0ea52b4a9a61 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.798774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 796/1603] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:08 2022] PUT /identity/v3/projects/b807736ef6c54be2b4715f32a4cfea57/users/ea2f6b804e164dbcb4f67926f23fdd59/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:08.806476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e94889-d990-4c48-9ced-f1d3673cd153 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.837466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d19e1-1ec4-4d4a-b027-0ea52b4a9a61 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.862543 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e94889-d990-4c48-9ced-f1d3673cd153 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:08.867759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261914b-0f6b-462e-837e-4509aed11d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.868413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261914b-0f6b-462e-837e-4509aed11d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.868693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261914b-0f6b-462e-837e-4509aed11d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.868959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261914b-0f6b-462e-837e-4509aed11d62 admin admin] PATH_INFO: `/v3/projects/48c466a9c78b46db90b2c979cf885f43/users/ae3824cef26941278065945a8ee37d32/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.883844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261914b-0f6b-462e-837e-4509aed11d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c466a9c78b46db90b2c979cf885f43, user_id=ae3824cef26941278065945a8ee37d32, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.886535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261914b-0f6b-462e-837e-4509aed11d62 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.895764 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a218c-a5a9-4aa2-b3c6-2fe3366bd2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:08.896465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a218c-a5a9-4aa2-b3c6-2fe3366bd2ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:08.896745 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a218c-a5a9-4aa2-b3c6-2fe3366bd2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:08.897020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a218c-a5a9-4aa2-b3c6-2fe3366bd2ee admin admin] PATH_INFO: `/v3/projects/f143f731c6cc4b38ba979827cd390f77/users/5a85cff936184b5d94375fd103493950/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:08.911418 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a218c-a5a9-4aa2-b3c6-2fe3366bd2ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f143f731c6cc4b38ba979827cd390f77, user_id=5a85cff936184b5d94375fd103493950, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:08.914350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a218c-a5a9-4aa2-b3c6-2fe3366bd2ee admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:08.980348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 797/1604] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:08 2022] PUT /identity/v3/projects/48c466a9c78b46db90b2c979cf885f43/users/ae3824cef26941278065945a8ee37d32/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:08.981256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261914b-0f6b-462e-837e-4509aed11d62 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:08.990295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 808/1605] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:08 2022] PUT /identity/v3/projects/f143f731c6cc4b38ba979827cd390f77/users/5a85cff936184b5d94375fd103493950/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:08.991839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a218c-a5a9-4aa2-b3c6-2fe3366bd2ee admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.048942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261914b-0f6b-462e-837e-4509aed11d62 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.053493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb86300-e798-4232-bb57-f07b2f038147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.054645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb86300-e798-4232-bb57-f07b2f038147 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.054922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a218c-a5a9-4aa2-b3c6-2fe3366bd2ee admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.055027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb86300-e798-4232-bb57-f07b2f038147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.055219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb86300-e798-4232-bb57-f07b2f038147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.055901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb86300-e798-4232-bb57-f07b2f038147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.057453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb86300-e798-4232-bb57-f07b2f038147 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.059530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2eac8-8cd2-43cf-b708-e21890988202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.060181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2eac8-8cd2-43cf-b708-e21890988202 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.060462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2eac8-8cd2-43cf-b708-e21890988202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.060734 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2eac8-8cd2-43cf-b708-e21890988202 admin admin] PATH_INFO: `/v3/projects/45838a0f741e485e8f424375c6d0d489/users/393fddf9c75447fab334b84694dcf4f1/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.067269 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 798/1606] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:09.068446 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677e4c-904f-4ef6-ab49-e919d98f5c95 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.068677 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:09.068785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677e4c-904f-4ef6-ab49-e919d98f5c95 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.068997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677e4c-904f-4ef6-ab49-e919d98f5c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.078099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2eac8-8cd2-43cf-b708-e21890988202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45838a0f741e485e8f424375c6d0d489, user_id=393fddf9c75447fab334b84694dcf4f1, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.080870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2eac8-8cd2-43cf-b708-e21890988202 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.095714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-db677e4c-904f-4ef6-ab49-e919d98f5c95 None None] MFA Rules not processed for user `47aec2c802084ca6b949b757692997bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:09.096925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-db677e4c-904f-4ef6-ab49-e919d98f5c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:09.117976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:09.145749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 809/1607] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:08 2022] PUT /identity/v3/projects/45838a0f741e485e8f424375c6d0d489/users/393fddf9c75447fab334b84694dcf4f1/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:09.147016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042f91d-e163-4df8-a383-fab56608ba0f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.147390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042f91d-e163-4df8-a383-fab56608ba0f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.147724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042f91d-e163-4df8-a383-fab56608ba0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.165389 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 799/1608] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:09 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:09.166626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65487e4-0df4-4de2-89b6-73a62fdf9fa8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.167043 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65487e4-0df4-4de2-89b6-73a62fdf9fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.167319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65487e4-0df4-4de2-89b6-73a62fdf9fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.176293 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f042f91d-e163-4df8-a383-fab56608ba0f None None] MFA Rules not processed for user `5e123f7323f3463fa812e51edf28d2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:09.180380 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f042f91d-e163-4df8-a383-fab56608ba0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:09.195223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f65487e4-0df4-4de2-89b6-73a62fdf9fa8 None None] MFA Rules not processed for user `755487a8ea3d48a58983efea5c7a1d62`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:09.196408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f65487e4-0df4-4de2-89b6-73a62fdf9fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:09.242190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 810/1609] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:09.243746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018aac-4154-4468-bdf9-da57f59b709d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.244031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018aac-4154-4468-bdf9-da57f59b709d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.244298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018aac-4154-4468-bdf9-da57f59b709d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.261346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 800/1610] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:09.261944 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e5018aac-4154-4468-bdf9-da57f59b709d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:09.262385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be4615-4810-46f6-b8c6-c6b351d2d54b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.262665 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be4615-4810-46f6-b8c6-c6b351d2d54b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.262956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be4615-4810-46f6-b8c6-c6b351d2d54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.263171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e5018aac-4154-4468-bdf9-da57f59b709d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:09.281234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-65be4615-4810-46f6-b8c6-c6b351d2d54b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:09.282415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-65be4615-4810-46f6-b8c6-c6b351d2d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:09.323782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 811/1611] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:09.325070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456bb6d-d77c-4f93-87d5-13b8a4b180c1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.325351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456bb6d-d77c-4f93-87d5-13b8a4b180c1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.325619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456bb6d-d77c-4f93-87d5-13b8a4b180c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.343800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f456bb6d-d77c-4f93-87d5-13b8a4b180c1 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:09.344524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 801/1612] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:09.345372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be4615-4810-46f6-b8c6-c6b351d2d54b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.348044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f456bb6d-d77c-4f93-87d5-13b8a4b180c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:09.355102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be4615-4810-46f6-b8c6-c6b351d2d54b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.356313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 812/1613] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:09.357503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c811a-dd04-40b3-b0c1-e766d2b1c97c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.357781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c811a-dd04-40b3-b0c1-e766d2b1c97c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.358062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c811a-dd04-40b3-b0c1-e766d2b1c97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.362723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dfaf1-2105-47d0-8fa8-3a2d542b2d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.363388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dfaf1-2105-47d0-8fa8-3a2d542b2d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.363672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dfaf1-2105-47d0-8fa8-3a2d542b2d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.363951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dfaf1-2105-47d0-8fa8-3a2d542b2d10 admin admin] PATH_INFO: `/v3/projects/eb52d54cfb144457a71c78339f74d936/users/0304c4e7f6b34b44b3fcbdca0d8e9b6e/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.378768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-292c811a-dd04-40b3-b0c1-e766d2b1c97c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:09.379915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dfaf1-2105-47d0-8fa8-3a2d542b2d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb52d54cfb144457a71c78339f74d936, user_id=0304c4e7f6b34b44b3fcbdca0d8e9b6e, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.380051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-292c811a-dd04-40b3-b0c1-e766d2b1c97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:09.382630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dfaf1-2105-47d0-8fa8-3a2d542b2d10 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.387584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 813/1614] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:09.388370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c811a-dd04-40b3-b0c1-e766d2b1c97c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.398238 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c811a-dd04-40b3-b0c1-e766d2b1c97c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.402900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb1565-9979-4c31-b173-4954799dcf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.403501 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb1565-9979-4c31-b173-4954799dcf26 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.403819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb1565-9979-4c31-b173-4954799dcf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.404092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb1565-9979-4c31-b173-4954799dcf26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.404716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb1565-9979-4c31-b173-4954799dcf26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.406606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb1565-9979-4c31-b173-4954799dcf26 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.437760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 802/1615] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:09 2022] PUT /identity/v3/projects/eb52d54cfb144457a71c78339f74d936/users/0304c4e7f6b34b44b3fcbdca0d8e9b6e/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:09.438770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dfaf1-2105-47d0-8fa8-3a2d542b2d10 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.469338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:09.485186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 814/1616] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:09 2022] POST /identity/v3/projects => generated 403 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:09.486024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb1565-9979-4c31-b173-4954799dcf26 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.506925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dfaf1-2105-47d0-8fa8-3a2d542b2d10 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.533473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:09.543621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb1565-9979-4c31-b173-4954799dcf26 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.547179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160b8c7-993b-4f7a-b721-7de8985242bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.547780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160b8c7-993b-4f7a-b721-7de8985242bc service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.548058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160b8c7-993b-4f7a-b721-7de8985242bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.548331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160b8c7-993b-4f7a-b721-7de8985242bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.563459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b91744-c03b-48f7-a081-9f2b904ef525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.564192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91744-c03b-48f7-a081-9f2b904ef525 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.564472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91744-c03b-48f7-a081-9f2b904ef525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.564746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91744-c03b-48f7-a081-9f2b904ef525 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.565381 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b91744-c03b-48f7-a081-9f2b904ef525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.567399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b91744-c03b-48f7-a081-9f2b904ef525 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.606303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160b8c7-993b-4f7a-b721-7de8985242bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.607839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160b8c7-993b-4f7a-b721-7de8985242bc service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.617215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 815/1617] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:09.618003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160b8c7-993b-4f7a-b721-7de8985242bc service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.634104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 803/1618] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:09 2022] POST /identity/v3/projects => generated 373 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:09.635206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b91744-c03b-48f7-a081-9f2b904ef525 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.678724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160b8c7-993b-4f7a-b721-7de8985242bc service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.695244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b91744-c03b-48f7-a081-9f2b904ef525 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.698899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61c6e9-db65-41de-8cb1-765f0b222ee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.699510 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61c6e9-db65-41de-8cb1-765f0b222ee9 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.699787 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61c6e9-db65-41de-8cb1-765f0b222ee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.700055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61c6e9-db65-41de-8cb1-765f0b222ee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.743509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed34070-bbd5-4e5f-97a2-06d9f52de570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.743743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed34070-bbd5-4e5f-97a2-06d9f52de570 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.744024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed34070-bbd5-4e5f-97a2-06d9f52de570 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.744301 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed34070-bbd5-4e5f-97a2-06d9f52de570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.777780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61c6e9-db65-41de-8cb1-765f0b222ee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.779238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61c6e9-db65-41de-8cb1-765f0b222ee9 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.790128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 804/1619] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:09.790986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61c6e9-db65-41de-8cb1-765f0b222ee9 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.800091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61c6e9-db65-41de-8cb1-765f0b222ee9 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.805040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f645e-7efe-495b-8d24-0541e180bd6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.805747 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f645e-7efe-495b-8d24-0541e180bd6e service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.806032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f645e-7efe-495b-8d24-0541e180bd6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.806499 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f645e-7efe-495b-8d24-0541e180bd6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.816089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f645e-7efe-495b-8d24-0541e180bd6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.817715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f645e-7efe-495b-8d24-0541e180bd6e service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.818079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed34070-bbd5-4e5f-97a2-06d9f52de570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.819697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed34070-bbd5-4e5f-97a2-06d9f52de570 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.822124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:09.831066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 805/1620] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:09.833240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f645e-7efe-495b-8d24-0541e180bd6e service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.834313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 816/1621] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:09.835232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed34070-bbd5-4e5f-97a2-06d9f52de570 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.840493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:09.842243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f645e-7efe-495b-8d24-0541e180bd6e service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.846221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed34070-bbd5-4e5f-97a2-06d9f52de570 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.846405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1376787-86e3-4d53-a8bb-83373fd37199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.847051 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1376787-86e3-4d53-a8bb-83373fd37199 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.847331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1376787-86e3-4d53-a8bb-83373fd37199 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.847747 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1376787-86e3-4d53-a8bb-83373fd37199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.856062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19817f46-6fbc-4b02-a11d-880cc04cdcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.856814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19817f46-6fbc-4b02-a11d-880cc04cdcf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.857090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19817f46-6fbc-4b02-a11d-880cc04cdcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.857369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19817f46-6fbc-4b02-a11d-880cc04cdcf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.858910 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19817f46-6fbc-4b02-a11d-880cc04cdcf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.861040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19817f46-6fbc-4b02-a11d-880cc04cdcf6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.864220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1376787-86e3-4d53-a8bb-83373fd37199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.865781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1376787-86e3-4d53-a8bb-83373fd37199 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.880732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 817/1622] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:09.880940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 806/1623] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:09.880940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1376787-86e3-4d53-a8bb-83373fd37199 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.881118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19817f46-6fbc-4b02-a11d-880cc04cdcf6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:09.889646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1376787-86e3-4d53-a8bb-83373fd37199 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.896045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a18e1-cb84-49de-a091-dc74f8848b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.896920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a18e1-cb84-49de-a091-dc74f8848b99 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.897308 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a18e1-cb84-49de-a091-dc74f8848b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.897669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a18e1-cb84-49de-a091-dc74f8848b99 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.898399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a18e1-cb84-49de-a091-dc74f8848b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.899038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19817f46-6fbc-4b02-a11d-880cc04cdcf6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:09.900882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a18e1-cb84-49de-a091-dc74f8848b99 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:09.906123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0846aa-43bb-4092-a0d1-d5ef3a3e0f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:09.906974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0846aa-43bb-4092-a0d1-d5ef3a3e0f0c admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:09.907272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0846aa-43bb-4092-a0d1-d5ef3a3e0f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:09.907820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0846aa-43bb-4092-a0d1-d5ef3a3e0f0c admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:09.908575 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0846aa-43bb-4092-a0d1-d5ef3a3e0f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:09.910719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0846aa-43bb-4092-a0d1-d5ef3a3e0f0c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:10.037058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 818/1624] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:09 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:10.037350 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 807/1625] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:09 2022] POST /identity/v3/users => generated 308 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:10.038174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a18e1-cb84-49de-a091-dc74f8848b99 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:10.044159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0846aa-43bb-4092-a0d1-d5ef3a3e0f0c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:10.053329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a18e1-cb84-49de-a091-dc74f8848b99 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:10.056688 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0846aa-43bb-4092-a0d1-d5ef3a3e0f0c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:10.061832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb51115-8cb6-472f-a2cb-bb2d0dcea875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:10.062639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb51115-8cb6-472f-a2cb-bb2d0dcea875 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.063032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb51115-8cb6-472f-a2cb-bb2d0dcea875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.067025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb51115-8cb6-472f-a2cb-bb2d0dcea875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.067025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb51115-8cb6-472f-a2cb-bb2d0dcea875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:10.067025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb51115-8cb6-472f-a2cb-bb2d0dcea875 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:10.075481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7660ae-219c-4fbb-ad78-907404c1309e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:10.076218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7660ae-219c-4fbb-ad78-907404c1309e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.076441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7660ae-219c-4fbb-ad78-907404c1309e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.076711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7660ae-219c-4fbb-ad78-907404c1309e admin admin] PATH_INFO: `/v3/projects/eb52d54cfb144457a71c78339f74d936/users/0304c4e7f6b34b44b3fcbdca0d8e9b6e/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.082938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 819/1626] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:10.082938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb51115-8cb6-472f-a2cb-bb2d0dcea875 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:10.083915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7660ae-219c-4fbb-ad78-907404c1309e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb52d54cfb144457a71c78339f74d936, user_id=0304c4e7f6b34b44b3fcbdca0d8e9b6e, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:10.086301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7660ae-219c-4fbb-ad78-907404c1309e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:10.093157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb51115-8cb6-472f-a2cb-bb2d0dcea875 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:10.098052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed418b50-5f85-46e1-a023-7f751be5def5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:10.098569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed418b50-5f85-46e1-a023-7f751be5def5 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.098906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed418b50-5f85-46e1-a023-7f751be5def5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.099179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed418b50-5f85-46e1-a023-7f751be5def5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.099852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed418b50-5f85-46e1-a023-7f751be5def5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:10.101995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed418b50-5f85-46e1-a023-7f751be5def5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:10.115869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 808/1627] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:10 2022] PUT /identity/v3/projects/eb52d54cfb144457a71c78339f74d936/users/0304c4e7f6b34b44b3fcbdca0d8e9b6e/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:10.118751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 820/1628] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:10.119084 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7660ae-219c-4fbb-ad78-907404c1309e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:10.119673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed418b50-5f85-46e1-a023-7f751be5def5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:10.130857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed418b50-5f85-46e1-a023-7f751be5def5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:10.210604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7660ae-219c-4fbb-ad78-907404c1309e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:10.213450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454245b-de58-4826-b24c-0dcb065ba820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:10.214127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454245b-de58-4826-b24c-0dcb065ba820 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.214738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454245b-de58-4826-b24c-0dcb065ba820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.214738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454245b-de58-4826-b24c-0dcb065ba820 admin admin] PATH_INFO: `/v3/projects/59a5298f67954a85ba92d32efd9dc7f6/users/5b58a75084aa4203a5db2667caa840a4/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.216244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff614f3-eaff-405c-a47c-ebd429215d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:10.217323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff614f3-eaff-405c-a47c-ebd429215d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.217458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff614f3-eaff-405c-a47c-ebd429215d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.217775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff614f3-eaff-405c-a47c-ebd429215d9c admin admin] PATH_INFO: `/v3/projects/1885aea0d2324b9cbee2b441339b0a15/users/d1cef61d4f2f4bc1b72af0cf591f8231/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.232173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454245b-de58-4826-b24c-0dcb065ba820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a5298f67954a85ba92d32efd9dc7f6, user_id=5b58a75084aa4203a5db2667caa840a4, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:10.234533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff614f3-eaff-405c-a47c-ebd429215d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1885aea0d2324b9cbee2b441339b0a15, user_id=d1cef61d4f2f4bc1b72af0cf591f8231, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:10.235031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454245b-de58-4826-b24c-0dcb065ba820 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:10.237439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff614f3-eaff-405c-a47c-ebd429215d9c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:10.329279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 821/1629] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:10 2022] PUT /identity/v3/projects/59a5298f67954a85ba92d32efd9dc7f6/users/5b58a75084aa4203a5db2667caa840a4/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:10.333867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbdf73-a722-4346-9393-65dcbdbed70d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.333867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbdf73-a722-4346-9393-65dcbdbed70d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.333867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbdf73-a722-4346-9393-65dcbdbed70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.359442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-24cbdf73-a722-4346-9393-65dcbdbed70d None None] MFA Rules not processed for user `0304c4e7f6b34b44b3fcbdca0d8e9b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:10.360698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-24cbdf73-a722-4346-9393-65dcbdbed70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:10.374010 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 809/1630] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:10 2022] PUT /identity/v3/projects/1885aea0d2324b9cbee2b441339b0a15/users/d1cef61d4f2f4bc1b72af0cf591f8231/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:10.375336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff614f3-eaff-405c-a47c-ebd429215d9c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:10.451300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff614f3-eaff-405c-a47c-ebd429215d9c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:10.451938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 822/1631] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:10.452674 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbdf73-a722-4346-9393-65dcbdbed70d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:10.457062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c5217-fd9d-4d76-828f-3a87bbcdcab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:10.457972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c5217-fd9d-4d76-828f-3a87bbcdcab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.458315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c5217-fd9d-4d76-828f-3a87bbcdcab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.458710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c5217-fd9d-4d76-828f-3a87bbcdcab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.460360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c5217-fd9d-4d76-828f-3a87bbcdcab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:10.462040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c5217-fd9d-4d76-828f-3a87bbcdcab5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:10.477412 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:10.486737 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 810/1632] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:10.488159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c5217-fd9d-4d76-828f-3a87bbcdcab5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:10.547517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:10.565131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c5217-fd9d-4d76-828f-3a87bbcdcab5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:10.572822 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d61e4-38ed-4346-8048-67f888dee767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:10.573653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d61e4-38ed-4346-8048-67f888dee767 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.574035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d61e4-38ed-4346-8048-67f888dee767 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.574294 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d61e4-38ed-4346-8048-67f888dee767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.580891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbdf73-a722-4346-9393-65dcbdbed70d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:10.587288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ced542-ba4b-4a5c-a504-79ca08fe5808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:10.588081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ced542-ba4b-4a5c-a504-79ca08fe5808 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.588458 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ced542-ba4b-4a5c-a504-79ca08fe5808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.588827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ced542-ba4b-4a5c-a504-79ca08fe5808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.601294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ced542-ba4b-4a5c-a504-79ca08fe5808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:10.608466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ced542-ba4b-4a5c-a504-79ca08fe5808 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:10.629025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 823/1633] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:10.636687 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ced542-ba4b-4a5c-a504-79ca08fe5808 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:10.655892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ced542-ba4b-4a5c-a504-79ca08fe5808 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:10.665921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221eaafd-4eba-44fb-8a47-45e26e631ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:10.666605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221eaafd-4eba-44fb-8a47-45e26e631ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.666915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221eaafd-4eba-44fb-8a47-45e26e631ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.667191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221eaafd-4eba-44fb-8a47-45e26e631ff7 admin admin] PATH_INFO: `/v3/projects/59a5298f67954a85ba92d32efd9dc7f6/users/5b58a75084aa4203a5db2667caa840a4/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.672610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221eaafd-4eba-44fb-8a47-45e26e631ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a5298f67954a85ba92d32efd9dc7f6, user_id=5b58a75084aa4203a5db2667caa840a4, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:10.675575 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221eaafd-4eba-44fb-8a47-45e26e631ff7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:10.683520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d61e4-38ed-4346-8048-67f888dee767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:10.684850 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d61e4-38ed-4346-8048-67f888dee767 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:10.703200 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 824/1634] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:10 2022] PUT /identity/v3/projects/59a5298f67954a85ba92d32efd9dc7f6/users/5b58a75084aa4203a5db2667caa840a4/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:10.703200 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221eaafd-4eba-44fb-8a47-45e26e631ff7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:10.707432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 811/1635] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:10.710710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c196d-8eab-4b02-99ca-683945f98611 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.715127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c196d-8eab-4b02-99ca-683945f98611 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.721070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c196d-8eab-4b02-99ca-683945f98611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.776995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-366c196d-8eab-4b02-99ca-683945f98611 None None] MFA Rules not processed for user `5b58a75084aa4203a5db2667caa840a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:10.778538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-366c196d-8eab-4b02-99ca-683945f98611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:10.810073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221eaafd-4eba-44fb-8a47-45e26e631ff7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:10.815437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9756a4-071d-428c-96d9-307a0a56340f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:10.816296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9756a4-071d-428c-96d9-307a0a56340f admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.816748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9756a4-071d-428c-96d9-307a0a56340f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.817181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9756a4-071d-428c-96d9-307a0a56340f admin admin] PATH_INFO: `/v3/projects/1885aea0d2324b9cbee2b441339b0a15/users/d1cef61d4f2f4bc1b72af0cf591f8231/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.819437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9756a4-071d-428c-96d9-307a0a56340f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1885aea0d2324b9cbee2b441339b0a15, user_id=d1cef61d4f2f4bc1b72af0cf591f8231, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:10.823582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9756a4-071d-428c-96d9-307a0a56340f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:10.837367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 825/1636] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:10 2022] PUT /identity/v3/projects/1885aea0d2324b9cbee2b441339b0a15/users/d1cef61d4f2f4bc1b72af0cf591f8231/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:10.848189 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5d7b8-5c3c-4c04-b25a-6e46fbae9928 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:10.848492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5d7b8-5c3c-4c04-b25a-6e46fbae9928 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:10.848771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5d7b8-5c3c-4c04-b25a-6e46fbae9928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:10.883936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 812/1637] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:10 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:10.888840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3ff5d7b8-5c3c-4c04-b25a-6e46fbae9928 None None] MFA Rules not processed for user `d1cef61d4f2f4bc1b72af0cf591f8231`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:10.890312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3ff5d7b8-5c3c-4c04-b25a-6e46fbae9928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:10.893363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c196d-8eab-4b02-99ca-683945f98611 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:10.963437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c196d-8eab-4b02-99ca-683945f98611 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:10.982468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 826/1638] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:11.015458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4506e39-794a-44cd-ab5a-e16ac700d1c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:11.016058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4506e39-794a-44cd-ab5a-e16ac700d1c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:11.016338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4506e39-794a-44cd-ab5a-e16ac700d1c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:11.016611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4506e39-794a-44cd-ab5a-e16ac700d1c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:11.035992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5d7b8-5c3c-4c04-b25a-6e46fbae9928 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:11.046574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5d7b8-5c3c-4c04-b25a-6e46fbae9928 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:11.053061 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b644d34-a8ec-4c33-8660-b613c5b73851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:11.053061 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b644d34-a8ec-4c33-8660-b613c5b73851 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:11.053061 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b644d34-a8ec-4c33-8660-b613c5b73851 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:11.053061 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b644d34-a8ec-4c33-8660-b613c5b73851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:11.062192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b644d34-a8ec-4c33-8660-b613c5b73851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:11.063617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b644d34-a8ec-4c33-8660-b613c5b73851 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:11.089627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 827/1639] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:11.096813 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4506e39-794a-44cd-ab5a-e16ac700d1c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:11.098433 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4506e39-794a-44cd-ab5a-e16ac700d1c2 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:11.110775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 813/1640] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:10 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:05:11.242433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b644d34-a8ec-4c33-8660-b613c5b73851 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:11.249601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b644d34-a8ec-4c33-8660-b613c5b73851 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:11.272432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb493-207b-4bd4-a870-9d906fc976ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:11.272937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb493-207b-4bd4-a870-9d906fc976ef service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:11.273386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb493-207b-4bd4-a870-9d906fc976ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:11.273665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb493-207b-4bd4-a870-9d906fc976ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:11.278154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:11.287050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:11.360158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84eb493-207b-4bd4-a870-9d906fc976ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:11.361706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84eb493-207b-4bd4-a870-9d906fc976ef service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:11.377365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 828/1641] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:11.458534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4506e39-794a-44cd-ab5a-e16ac700d1c2 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:11.469013 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4506e39-794a-44cd-ab5a-e16ac700d1c2 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:11.474491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554310a-10f9-4e09-8dfc-e908525978e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:11.475473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554310a-10f9-4e09-8dfc-e908525978e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:11.480793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554310a-10f9-4e09-8dfc-e908525978e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:11.480793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554310a-10f9-4e09-8dfc-e908525978e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:11.565718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554310a-10f9-4e09-8dfc-e908525978e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:11.567203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554310a-10f9-4e09-8dfc-e908525978e3 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:11.582103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 814/1642] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:12.238411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb493-207b-4bd4-a870-9d906fc976ef service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:12.328264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb493-207b-4bd4-a870-9d906fc976ef service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:12.333824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875e4ce-0340-4d95-8586-50db380957a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:12.334565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875e4ce-0340-4d95-8586-50db380957a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:12.335027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875e4ce-0340-4d95-8586-50db380957a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:12.335498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875e4ce-0340-4d95-8586-50db380957a7 admin admin] PATH_INFO: `/v3/users/47aec2c802084ca6b949b757692997bc` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:12.336971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875e4ce-0340-4d95-8586-50db380957a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47aec2c802084ca6b949b757692997bc)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:12.340992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875e4ce-0340-4d95-8586-50db380957a7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:12.456844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7a26-19ef-4245-8354-119f664c4d5f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:12.457395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7a26-19ef-4245-8354-119f664c4d5f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:12.457395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7a26-19ef-4245-8354-119f664c4d5f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:12.464959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 815/1643] 10.222.0.59 () {54 vars in 962 bytes} [Fri Sep 16 08:05:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 08:05:12.472067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba57f00-2a38-4e3a-b649-d1f15cd34689 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:12.472356 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba57f00-2a38-4e3a-b649-d1f15cd34689 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:12.472783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba57f00-2a38-4e3a-b649-d1f15cd34689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:12.578795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-aba57f00-2a38-4e3a-b649-d1f15cd34689 None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:12.580332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-aba57f00-2a38-4e3a-b649-d1f15cd34689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:12.651967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a875e4ce-0340-4d95-8586-50db380957a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47aec2c802084ca6b949b757692997bc'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:12.677041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a875e4ce-0340-4d95-8586-50db380957a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47aec2c802084ca6b949b757692997bc'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:12.690963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a875e4ce-0340-4d95-8586-50db380957a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47aec2c802084ca6b949b757692997bc'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:12.696272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:12.731581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 816/1644] 10.222.0.59 () {58 vars in 1059 bytes} [Fri Sep 16 08:05:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:12.742663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a70c2-f5d6-4664-bd46-2baa0db947a7 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:12.745110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a70c2-f5d6-4664-bd46-2baa0db947a7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:12.745110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a70c2-f5d6-4664-bd46-2baa0db947a7 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:12.746221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 817/1645] 10.222.0.59 () {54 vars in 912 bytes} [Fri Sep 16 08:05:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:05:12.752510 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a875e4ce-0340-4d95-8586-50db380957a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47aec2c802084ca6b949b757692997bc'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:12.753353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a875e4ce-0340-4d95-8586-50db380957a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47aec2c802084ca6b949b757692997bc'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:12.773658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 829/1646] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:12 2022] DELETE /identity/v3/users/47aec2c802084ca6b949b757692997bc => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:12.775255 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd098527-dab3-4eea-af5c-d60a89f50073 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:12.775717 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd098527-dab3-4eea-af5c-d60a89f50073 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:12.776152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd098527-dab3-4eea-af5c-d60a89f50073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:12.779126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3dd28-8674-4de3-8d4b-0b89665194e9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:12.780258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3dd28-8674-4de3-8d4b-0b89665194e9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:12.781871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3dd28-8674-4de3-8d4b-0b89665194e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:12.826923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fd098527-dab3-4eea-af5c-d60a89f50073 None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:12.829200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fd098527-dab3-4eea-af5c-d60a89f50073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:12.830392 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-11e3dd28-8674-4de3-8d4b-0b89665194e9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:12.836866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-11e3dd28-8674-4de3-8d4b-0b89665194e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:12.884871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:12.979103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 830/1647] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:12.993484 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 818/1648] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:05:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:13.008328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3dd28-8674-4de3-8d4b-0b89665194e9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:13.010060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-11e3dd28-8674-4de3-8d4b-0b89665194e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:13.052711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd098527-dab3-4eea-af5c-d60a89f50073 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:13.055134 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fd098527-dab3-4eea-af5c-d60a89f50073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:13.126098 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3dd28-8674-4de3-8d4b-0b89665194e9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:13.131183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8788-4d69-44b0-a9e4-e433c2d3924a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:13.131977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8788-4d69-44b0-a9e4-e433c2d3924a service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:13.132676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8788-4d69-44b0-a9e4-e433c2d3924a service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:13.133174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8788-4d69-44b0-a9e4-e433c2d3924a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:13.144162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd098527-dab3-4eea-af5c-d60a89f50073 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:13.146627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c8788-4d69-44b0-a9e4-e433c2d3924a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:13.148857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d6074-142c-4161-95d9-db4bd233cd7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:13.149658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c8788-4d69-44b0-a9e4-e433c2d3924a service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:13.149792 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d6074-142c-4161-95d9-db4bd233cd7f service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:13.150075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d6074-142c-4161-95d9-db4bd233cd7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:13.150418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d6074-142c-4161-95d9-db4bd233cd7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:13.166958 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d6074-142c-4161-95d9-db4bd233cd7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:13.171001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d6074-142c-4161-95d9-db4bd233cd7f service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:13.188820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 831/1649] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:13.197461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 819/1650] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:05:15.006827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8788-4d69-44b0-a9e4-e433c2d3924a service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:15.009415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a41c8788-4d69-44b0-a9e4-e433c2d3924a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:15.037690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8788-4d69-44b0-a9e4-e433c2d3924a service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:15.043324 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:15.049488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:15.049488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:15.049488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] PATH_INFO: `/v3/projects/8f789d3ecccd4440a05d7bfd944c98bd` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:15.049488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f789d3ecccd4440a05d7bfd944c98bd)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:15.049488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:05:15.049488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 16 08:05:15.049488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: Sep 16 08:05:15.049949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:15.331753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f789d3ecccd4440a05d7bfd944c98bd', 'request_id': 'req-8db6a371-ab94-4b51-b0b8-e9ea23846b25'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:15.381037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f789d3ecccd4440a05d7bfd944c98bd', 'request_id': 'req-8db6a371-ab94-4b51-b0b8-e9ea23846b25'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:15.417255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f789d3ecccd4440a05d7bfd944c98bd', 'request_id': 'req-8db6a371-ab94-4b51-b0b8-e9ea23846b25'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:15.567297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] The token cache is being invalidate because project 8f789d3ecccd4440a05d7bfd944c98bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:15.567666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:15.574950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8db6a371-ab94-4b51-b0b8-e9ea23846b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:15.578065 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 832/1651] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:15 2022] DELETE /identity/v3/projects/8f789d3ecccd4440a05d7bfd944c98bd => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:15.582701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d6074-142c-4161-95d9-db4bd233cd7f service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:15.599920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d6074-142c-4161-95d9-db4bd233cd7f service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:15.898209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:15.898209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:15.898209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:15.898209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] PATH_INFO: `/v3/users/5b58a75084aa4203a5db2667caa840a4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:15.907691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b58a75084aa4203a5db2667caa840a4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:15.908774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 08:05:15.908774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 16 08:05:15.908774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: Sep 16 08:05:15.911722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:16.107024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:16.213880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4133310-047e-466b-89be-2533bb8df0d7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:16.214841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4133310-047e-466b-89be-2533bb8df0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:16.215297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4133310-047e-466b-89be-2533bb8df0d7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:16.217126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 833/1652] 10.222.0.59 () {54 vars in 962 bytes} [Fri Sep 16 08:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:05:16.231462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acb3c7-1234-4a9c-b201-0af690b1edf8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:16.231875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acb3c7-1234-4a9c-b201-0af690b1edf8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:16.232173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acb3c7-1234-4a9c-b201-0af690b1edf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:16.270184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b58a75084aa4203a5db2667caa840a4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:16.277703 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-71acb3c7-1234-4a9c-b201-0af690b1edf8 None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:16.279059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-71acb3c7-1234-4a9c-b201-0af690b1edf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:16.339415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b58a75084aa4203a5db2667caa840a4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:16.371203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b58a75084aa4203a5db2667caa840a4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:16.383361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:16.415159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 834/1653] 10.222.0.59 () {58 vars in 1059 bytes} [Fri Sep 16 08:05:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:16.433310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e8b0b-5e49-4663-a675-946d3a6948de None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:16.438199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e8b0b-5e49-4663-a675-946d3a6948de None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:16.438199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e8b0b-5e49-4663-a675-946d3a6948de None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:16.438199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 835/1654] 10.222.0.59 () {54 vars in 912 bytes} [Fri Sep 16 08:05:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:05:16.446232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306bb12-c7a1-4bda-af23-03e5659416c2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:16.447981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306bb12-c7a1-4bda-af23-03e5659416c2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:16.449951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306bb12-c7a1-4bda-af23-03e5659416c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:16.493854 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3306bb12-c7a1-4bda-af23-03e5659416c2 None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:16.495125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3306bb12-c7a1-4bda-af23-03e5659416c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:16.512667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b58a75084aa4203a5db2667caa840a4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:16.513680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b58a75084aa4203a5db2667caa840a4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:16.514987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 836/1655] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:05:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:16.525390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 820/1656] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:15 2022] DELETE /identity/v3/users/5b58a75084aa4203a5db2667caa840a4 => generated 0 bytes in 940 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:16.533127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306bb12-c7a1-4bda-af23-03e5659416c2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:16.539073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3306bb12-c7a1-4bda-af23-03e5659416c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:16.660296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306bb12-c7a1-4bda-af23-03e5659416c2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:16.816520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab46ddf-63bb-4cee-b1e6-8c4a9f598446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:16.822262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab46ddf-63bb-4cee-b1e6-8c4a9f598446 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:16.822871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab46ddf-63bb-4cee-b1e6-8c4a9f598446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:16.823991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab46ddf-63bb-4cee-b1e6-8c4a9f598446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:16.831897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cab46ddf-63bb-4cee-b1e6-8c4a9f598446 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:16.958332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab46ddf-63bb-4cee-b1e6-8c4a9f598446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:16.958332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab46ddf-63bb-4cee-b1e6-8c4a9f598446 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:16.975118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 837/1657] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:18.571385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:18.572977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:18.708874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f581e-ea57-48a8-8f51-63084d69b9ca admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:18.720726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:18.721648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:18.722044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:18.723045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] PATH_INFO: `/v3/projects/59a5298f67954a85ba92d32efd9dc7f6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:18.724855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a5298f67954a85ba92d32efd9dc7f6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:18.728265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:18.944751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a5298f67954a85ba92d32efd9dc7f6', 'request_id': 'req-0982ef39-0415-4cad-9e03-e6d159db3b44'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:18.976542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a5298f67954a85ba92d32efd9dc7f6', 'request_id': 'req-0982ef39-0415-4cad-9e03-e6d159db3b44'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:18.995776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a5298f67954a85ba92d32efd9dc7f6', 'request_id': 'req-0982ef39-0415-4cad-9e03-e6d159db3b44'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:19.159450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] The token cache is being invalidate because project 59a5298f67954a85ba92d32efd9dc7f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:19.160724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:19.162974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0982ef39-0415-4cad-9e03-e6d159db3b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:19.166020 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 821/1658] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:18 2022] DELETE /identity/v3/projects/59a5298f67954a85ba92d32efd9dc7f6 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:19.171911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683444b-0938-4f3f-bf3e-8e5c4cfead55 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:19.173104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683444b-0938-4f3f-bf3e-8e5c4cfead55 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:19.173629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683444b-0938-4f3f-bf3e-8e5c4cfead55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:19.227710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b683444b-0938-4f3f-bf3e-8e5c4cfead55 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:19.229353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b683444b-0938-4f3f-bf3e-8e5c4cfead55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:19.326531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:19.391579 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 838/1659] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:19.401753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aa1ee-e1c8-47c3-bc39-cc9cbbbf940f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:19.402038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aa1ee-e1c8-47c3-bc39-cc9cbbbf940f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:19.402338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aa1ee-e1c8-47c3-bc39-cc9cbbbf940f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:19.447031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4f0aa1ee-e1c8-47c3-bc39-cc9cbbbf940f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:19.448670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4f0aa1ee-e1c8-47c3-bc39-cc9cbbbf940f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:19.469840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 822/1660] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:19.478725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683444b-0938-4f3f-bf3e-8e5c4cfead55 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:19.492946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683444b-0938-4f3f-bf3e-8e5c4cfead55 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:19.504805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b4edf-38d3-4f60-b22f-9c7ea6e5ec80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:19.505444 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b4edf-38d3-4f60-b22f-9c7ea6e5ec80 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:19.505725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b4edf-38d3-4f60-b22f-9c7ea6e5ec80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:19.506052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b4edf-38d3-4f60-b22f-9c7ea6e5ec80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:19.507388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b4edf-38d3-4f60-b22f-9c7ea6e5ec80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:19.510971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b4edf-38d3-4f60-b22f-9c7ea6e5ec80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:19.510971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b4edf-38d3-4f60-b22f-9c7ea6e5ec80 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:19.523890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 839/1661] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:19.529841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aa1ee-e1c8-47c3-bc39-cc9cbbbf940f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:19.545106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aa1ee-e1c8-47c3-bc39-cc9cbbbf940f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:19.560479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d54b3-e149-41ff-9e9c-899514aaa1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:19.570461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d54b3-e149-41ff-9e9c-899514aaa1c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:19.570461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d54b3-e149-41ff-9e9c-899514aaa1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:19.570461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d54b3-e149-41ff-9e9c-899514aaa1c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:19.570461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d54b3-e149-41ff-9e9c-899514aaa1c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:19.572462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d54b3-e149-41ff-9e9c-899514aaa1c3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:19.706668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 823/1662] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:19 2022] POST /identity/v3/projects => generated 397 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:19.712297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b4edf-38d3-4f60-b22f-9c7ea6e5ec80 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:19.724892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b4edf-38d3-4f60-b22f-9c7ea6e5ec80 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:19.852943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d591ff-67a0-469f-bcfe-331b2277fe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:19.853590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d591ff-67a0-469f-bcfe-331b2277fe51 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:19.853876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d591ff-67a0-469f-bcfe-331b2277fe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:19.854147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d591ff-67a0-469f-bcfe-331b2277fe51 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:19.854958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d591ff-67a0-469f-bcfe-331b2277fe51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:19.856924 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d591ff-67a0-469f-bcfe-331b2277fe51 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:19.999456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 840/1663] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:19 2022] POST /identity/v3/users => generated 299 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:20.001822 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d54b3-e149-41ff-9e9c-899514aaa1c3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:20.019244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d54b3-e149-41ff-9e9c-899514aaa1c3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:20.029110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c0fee-f617-4f47-b054-180cb06db26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:20.030338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c0fee-f617-4f47-b054-180cb06db26e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:20.030338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c0fee-f617-4f47-b054-180cb06db26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:20.032185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c0fee-f617-4f47-b054-180cb06db26e admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:20.034754 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c0fee-f617-4f47-b054-180cb06db26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:20.039333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c0fee-f617-4f47-b054-180cb06db26e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:20.043545 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:20.073268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 824/1664] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:20.077396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d591ff-67a0-469f-bcfe-331b2277fe51 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:20.090125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d591ff-67a0-469f-bcfe-331b2277fe51 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:20.109508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9d2cb-8ff1-482d-a67f-4ef42c91c089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:20.110396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9d2cb-8ff1-482d-a67f-4ef42c91c089 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:20.110613 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9d2cb-8ff1-482d-a67f-4ef42c91c089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:20.110930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9d2cb-8ff1-482d-a67f-4ef42c91c089 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:20.111589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9d2cb-8ff1-482d-a67f-4ef42c91c089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:20.113211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9d2cb-8ff1-482d-a67f-4ef42c91c089 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:20.170233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 841/1665] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:20.176227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c0fee-f617-4f47-b054-180cb06db26e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:20.200118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c0fee-f617-4f47-b054-180cb06db26e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:20.214512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68d6e4-8de6-47d0-a65b-b24e4c87682e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:20.215256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68d6e4-8de6-47d0-a65b-b24e4c87682e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:20.216682 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68d6e4-8de6-47d0-a65b-b24e4c87682e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:20.223492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68d6e4-8de6-47d0-a65b-b24e4c87682e admin admin] PATH_INFO: `/v3/projects/8bd2d82a53b14363a6c82a6d4f2927e2/users/0a82437ccb64457b945085c3b57c7a9a/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:20.260091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68d6e4-8de6-47d0-a65b-b24e4c87682e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd2d82a53b14363a6c82a6d4f2927e2, user_id=0a82437ccb64457b945085c3b57c7a9a, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:20.262991 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68d6e4-8de6-47d0-a65b-b24e4c87682e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:20.378750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 825/1666] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:20 2022] PUT /identity/v3/projects/8bd2d82a53b14363a6c82a6d4f2927e2/users/0a82437ccb64457b945085c3b57c7a9a/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:20.383173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baab7cb-eebe-416c-83e3-bcf41364e80d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:20.383173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baab7cb-eebe-416c-83e3-bcf41364e80d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:20.383799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baab7cb-eebe-416c-83e3-bcf41364e80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:20.419890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4baab7cb-eebe-416c-83e3-bcf41364e80d None None] MFA Rules not processed for user `0a82437ccb64457b945085c3b57c7a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:20.421462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4baab7cb-eebe-416c-83e3-bcf41364e80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:20.552533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68d6e4-8de6-47d0-a65b-b24e4c87682e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:20.553198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 842/1667] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:20 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:20.575740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de33c3f-bb88-40ea-acdd-2a163b3e711d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:20.576270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de33c3f-bb88-40ea-acdd-2a163b3e711d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:20.576757 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de33c3f-bb88-40ea-acdd-2a163b3e711d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:20.666586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7de33c3f-bb88-40ea-acdd-2a163b3e711d None None] MFA Rules not processed for user `ea2f6b804e164dbcb4f67926f23fdd59`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:20.671333 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7de33c3f-bb88-40ea-acdd-2a163b3e711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:20.748924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68d6e4-8de6-47d0-a65b-b24e4c87682e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:20.758110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:20.764170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf37f3-6620-4af2-a0b4-e38bf24eebb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:20.764170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf37f3-6620-4af2-a0b4-e38bf24eebb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:20.764665 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf37f3-6620-4af2-a0b4-e38bf24eebb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:20.765019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf37f3-6620-4af2-a0b4-e38bf24eebb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:20.765738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf37f3-6620-4af2-a0b4-e38bf24eebb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:20.771331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf37f3-6620-4af2-a0b4-e38bf24eebb8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:20.862755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 843/1668] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:20.879806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de33c3f-bb88-40ea-acdd-2a163b3e711d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:20.881340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7de33c3f-bb88-40ea-acdd-2a163b3e711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:20.947517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 826/1669] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:20 2022] POST /identity/v3/projects => generated 401 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:20.963652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf37f3-6620-4af2-a0b4-e38bf24eebb8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:21.166728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de33c3f-bb88-40ea-acdd-2a163b3e711d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:21.169211 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf37f3-6620-4af2-a0b4-e38bf24eebb8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:21.175458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db860e-f6dc-435f-834d-dc345ba77231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:21.176297 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db860e-f6dc-435f-834d-dc345ba77231 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:21.176751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db860e-f6dc-435f-834d-dc345ba77231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:21.177162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db860e-f6dc-435f-834d-dc345ba77231 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:21.179398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db860e-f6dc-435f-834d-dc345ba77231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:21.181616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db860e-f6dc-435f-834d-dc345ba77231 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:21.294180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 827/1670] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:20 2022] POST /identity/v3/users => generated 307 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:21.295399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db860e-f6dc-435f-834d-dc345ba77231 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:21.318650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db860e-f6dc-435f-834d-dc345ba77231 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:21.326360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49fc60-a70f-4646-9254-2c95dfc3b3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:21.327565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49fc60-a70f-4646-9254-2c95dfc3b3db admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:21.327934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42040994-c09e-4ba1-ae19-7dcdcaa5faff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:21.328405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49fc60-a70f-4646-9254-2c95dfc3b3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:21.328580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42040994-c09e-4ba1-ae19-7dcdcaa5faff service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:21.328830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42040994-c09e-4ba1-ae19-7dcdcaa5faff service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:21.329102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42040994-c09e-4ba1-ae19-7dcdcaa5faff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:21.329463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49fc60-a70f-4646-9254-2c95dfc3b3db admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:21.330709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49fc60-a70f-4646-9254-2c95dfc3b3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:21.332950 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49fc60-a70f-4646-9254-2c95dfc3b3db admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:21.361709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 828/1671] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:05:21.363047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49fc60-a70f-4646-9254-2c95dfc3b3db admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:21.388110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49fc60-a70f-4646-9254-2c95dfc3b3db admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:21.393828 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a9659-55b7-4a6b-9cf6-b489ea322a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:21.394662 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a9659-55b7-4a6b-9cf6-b489ea322a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:21.395093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a9659-55b7-4a6b-9cf6-b489ea322a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:21.395476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a9659-55b7-4a6b-9cf6-b489ea322a6c admin admin] PATH_INFO: `/v3/projects/2df3d15d614f4de3b5ad652b8df17e2c/users/d2db04b6b2d0456bb7518a66fa631963/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:21.414602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a9659-55b7-4a6b-9cf6-b489ea322a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df3d15d614f4de3b5ad652b8df17e2c, user_id=d2db04b6b2d0456bb7518a66fa631963, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:21.422967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a9659-55b7-4a6b-9cf6-b489ea322a6c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:21.437909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:21.441638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42040994-c09e-4ba1-ae19-7dcdcaa5faff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:21.444772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42040994-c09e-4ba1-ae19-7dcdcaa5faff service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:21.461030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 844/1672] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:05:21.502589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 829/1673] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:21 2022] PUT /identity/v3/projects/2df3d15d614f4de3b5ad652b8df17e2c/users/d2db04b6b2d0456bb7518a66fa631963/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:21.507485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42040994-c09e-4ba1-ae19-7dcdcaa5faff service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:21.524984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42040994-c09e-4ba1-ae19-7dcdcaa5faff service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:21.638868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc67e6-4943-41cd-8825-5243106d5c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:21.639502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc67e6-4943-41cd-8825-5243106d5c9e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:21.639850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc67e6-4943-41cd-8825-5243106d5c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:21.640142 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc67e6-4943-41cd-8825-5243106d5c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:21.642798 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc67e6-4943-41cd-8825-5243106d5c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:21.644653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc67e6-4943-41cd-8825-5243106d5c9e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:21.690728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 845/1674] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:21.694009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a9659-55b7-4a6b-9cf6-b489ea322a6c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:21.719100 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a9659-55b7-4a6b-9cf6-b489ea322a6c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:21.735858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f2bf4-9bbd-4519-8d57-9ce8c7f29c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:21.736805 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f2bf4-9bbd-4519-8d57-9ce8c7f29c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:21.737219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f2bf4-9bbd-4519-8d57-9ce8c7f29c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:21.737614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f2bf4-9bbd-4519-8d57-9ce8c7f29c7e admin admin] PATH_INFO: `/v3/projects/2df3d15d614f4de3b5ad652b8df17e2c/users/d2db04b6b2d0456bb7518a66fa631963/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:21.739881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f2bf4-9bbd-4519-8d57-9ce8c7f29c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df3d15d614f4de3b5ad652b8df17e2c, user_id=d2db04b6b2d0456bb7518a66fa631963, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:21.746716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f2bf4-9bbd-4519-8d57-9ce8c7f29c7e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:21.774658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:21.787800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 830/1675] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:21 2022] PUT /identity/v3/projects/2df3d15d614f4de3b5ad652b8df17e2c/users/d2db04b6b2d0456bb7518a66fa631963/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:21.799491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e4e34-b447-4d99-8f31-2373ac5280b8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:21.800016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e4e34-b447-4d99-8f31-2373ac5280b8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:21.800397 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e4e34-b447-4d99-8f31-2373ac5280b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:21.835302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-588e4e34-b447-4d99-8f31-2373ac5280b8 None None] MFA Rules not processed for user `d2db04b6b2d0456bb7518a66fa631963`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:21.836671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-588e4e34-b447-4d99-8f31-2373ac5280b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:22.044531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 846/1676] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:21 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:22.054994 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f2bf4-9bbd-4519-8d57-9ce8c7f29c7e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:22.058047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3d1f2bf4-9bbd-4519-8d57-9ce8c7f29c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:22.204444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f2bf4-9bbd-4519-8d57-9ce8c7f29c7e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:22.217647 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358fab6-cee2-45bb-9314-665a23b9f3ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:22.221665 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358fab6-cee2-45bb-9314-665a23b9f3ca service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:22.221665 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358fab6-cee2-45bb-9314-665a23b9f3ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:22.221817 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358fab6-cee2-45bb-9314-665a23b9f3ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:22.250429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358fab6-cee2-45bb-9314-665a23b9f3ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:22.252359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358fab6-cee2-45bb-9314-665a23b9f3ca service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:22.302283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 831/1677] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:22 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:22.617015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e4e34-b447-4d99-8f31-2373ac5280b8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:22.634312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e4e34-b447-4d99-8f31-2373ac5280b8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:22.638354 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04827910-bd73-4908-9504-abc949ca4d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:22.639005 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04827910-bd73-4908-9504-abc949ca4d59 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:22.639287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04827910-bd73-4908-9504-abc949ca4d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:22.639563 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04827910-bd73-4908-9504-abc949ca4d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:22.772291 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04827910-bd73-4908-9504-abc949ca4d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:22.773991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04827910-bd73-4908-9504-abc949ca4d59 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:22.807055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 847/1678] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:22 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:24.976421 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcb469-3b9f-40de-b572-f3e7c703ad33 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:24.978411 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcb469-3b9f-40de-b572-f3e7c703ad33 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:24.978896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcb469-3b9f-40de-b572-f3e7c703ad33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:25.055902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-defcb469-3b9f-40de-b572-f3e7c703ad33 None None] MFA Rules not processed for user `ae3824cef26941278065945a8ee37d32`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:25.057352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-defcb469-3b9f-40de-b572-f3e7c703ad33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:25.274768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 832/1679] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:25.279968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e4c3b-0810-497a-b16c-c7d3243cbc1d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:25.280491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e4c3b-0810-497a-b16c-c7d3243cbc1d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:25.280491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e4c3b-0810-497a-b16c-c7d3243cbc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:25.338012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9d1e4c3b-0810-497a-b16c-c7d3243cbc1d None None] MFA Rules not processed for user `ae3824cef26941278065945a8ee37d32`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:25.339293 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9d1e4c3b-0810-497a-b16c-c7d3243cbc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:25.375819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 848/1680] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:25.392949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcb469-3b9f-40de-b572-f3e7c703ad33 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:25.392949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-defcb469-3b9f-40de-b572-f3e7c703ad33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:25.700054 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcb469-3b9f-40de-b572-f3e7c703ad33 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:25.744706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86506b9c-7e0b-4e76-8331-7692285554a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:25.745240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86506b9c-7e0b-4e76-8331-7692285554a4 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:25.745523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86506b9c-7e0b-4e76-8331-7692285554a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:25.745796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86506b9c-7e0b-4e76-8331-7692285554a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:25.780158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86506b9c-7e0b-4e76-8331-7692285554a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:25.781754 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86506b9c-7e0b-4e76-8331-7692285554a4 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:25.810197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 833/1681] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:26.681227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcfc6e-38b1-451e-b3c3-f1bd9e29a5f0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:26.682423 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcfc6e-38b1-451e-b3c3-f1bd9e29a5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:26.682893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcfc6e-38b1-451e-b3c3-f1bd9e29a5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:26.727682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:26.746390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b7dcfc6e-38b1-451e-b3c3-f1bd9e29a5f0 None None] MFA Rules not processed for user `5a85cff936184b5d94375fd103493950`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:26.751586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b7dcfc6e-38b1-451e-b3c3-f1bd9e29a5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:26.904312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 849/1682] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:26.914387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3da23-65a1-4995-b6ff-eabb78dcc315 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:26.914898 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3da23-65a1-4995-b6ff-eabb78dcc315 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:26.914985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3da23-65a1-4995-b6ff-eabb78dcc315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:26.946608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7ce3da23-65a1-4995-b6ff-eabb78dcc315 None None] MFA Rules not processed for user `5a85cff936184b5d94375fd103493950`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:26.948729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7ce3da23-65a1-4995-b6ff-eabb78dcc315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:26.974327 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76508933-2994-41ed-8c74-496500ccc85a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:26.974840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76508933-2994-41ed-8c74-496500ccc85a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:26.975321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76508933-2994-41ed-8c74-496500ccc85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:26.983917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 834/1683] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:26.988006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3da23-65a1-4995-b6ff-eabb78dcc315 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:27.014230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3da23-65a1-4995-b6ff-eabb78dcc315 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:27.027745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f823f37-b93b-4581-9631-f2f94531332d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:27.028969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f823f37-b93b-4581-9631-f2f94531332d service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:27.029221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f823f37-b93b-4581-9631-f2f94531332d service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:27.029498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f823f37-b93b-4581-9631-f2f94531332d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:27.053939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:27.066450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f823f37-b93b-4581-9631-f2f94531332d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:27.069844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f823f37-b93b-4581-9631-f2f94531332d service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:27.091315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-76508933-2994-41ed-8c74-496500ccc85a None None] MFA Rules not processed for user `393fddf9c75447fab334b84694dcf4f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:27.097013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-76508933-2994-41ed-8c74-496500ccc85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:27.104080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 835/1684] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:27.180897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574b3ef-580a-48ab-829a-a4102b34ad01 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:27.181221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574b3ef-580a-48ab-829a-a4102b34ad01 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:27.181459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574b3ef-580a-48ab-829a-a4102b34ad01 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:27.183529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 836/1685] 10.222.0.59 () {54 vars in 912 bytes} [Fri Sep 16 08:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:05:27.195758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f757579-7512-4474-a2d2-5df277234edb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:27.196040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f757579-7512-4474-a2d2-5df277234edb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:27.196787 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f757579-7512-4474-a2d2-5df277234edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:27.293342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7f757579-7512-4474-a2d2-5df277234edb None None] MFA Rules not processed for user `a29c4e53ac9a438e9701a70f3de3f32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:27.300786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7f757579-7512-4474-a2d2-5df277234edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:27.350018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 850/1686] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:27.533426 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 837/1687] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:05:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:27.636334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76508933-2994-41ed-8c74-496500ccc85a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:27.636334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-76508933-2994-41ed-8c74-496500ccc85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:27.660691 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76508933-2994-41ed-8c74-496500ccc85a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:27.677090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5388caa-08a1-4a71-99bc-2adb9d8346e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:27.677803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388caa-08a1-4a71-99bc-2adb9d8346e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:27.678181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388caa-08a1-4a71-99bc-2adb9d8346e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:27.679084 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388caa-08a1-4a71-99bc-2adb9d8346e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:27.704803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5388caa-08a1-4a71-99bc-2adb9d8346e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:27.710949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5388caa-08a1-4a71-99bc-2adb9d8346e2 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:27.742591 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 851/1688] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:28.123415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15693993-9f91-4cd0-807e-58ecb3c287fe None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:28.123845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15693993-9f91-4cd0-807e-58ecb3c287fe None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:28.124171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15693993-9f91-4cd0-807e-58ecb3c287fe None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:28.126376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 838/1689] 10.222.0.59 () {54 vars in 912 bytes} [Fri Sep 16 08:05:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:05:28.133647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82f749-ba69-4073-8288-80a6c6ee220c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:28.134160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82f749-ba69-4073-8288-80a6c6ee220c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:28.134638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82f749-ba69-4073-8288-80a6c6ee220c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:28.171706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7e82f749-ba69-4073-8288-80a6c6ee220c None None] MFA Rules not processed for user `a29c4e53ac9a438e9701a70f3de3f32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:28.173298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7e82f749-ba69-4073-8288-80a6c6ee220c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:28.184998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 852/1690] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:05:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:28.197559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15693993-9f91-4cd0-807e-58ecb3c287fe None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:28.203532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-15693993-9f91-4cd0-807e-58ecb3c287fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:28.221322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15693993-9f91-4cd0-807e-58ecb3c287fe None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:28.226045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b177f-79a8-41cc-b278-92214f341c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:28.227031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b177f-79a8-41cc-b278-92214f341c1e service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:28.227465 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b177f-79a8-41cc-b278-92214f341c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:28.227844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b177f-79a8-41cc-b278-92214f341c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:28.236768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b177f-79a8-41cc-b278-92214f341c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:28.240290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b177f-79a8-41cc-b278-92214f341c1e service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:28.264568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 839/1691] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:28.858087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258eac-2e8d-48fd-8d09-a69263c52f25 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:28.858619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258eac-2e8d-48fd-8d09-a69263c52f25 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:28.858728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258eac-2e8d-48fd-8d09-a69263c52f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:28.906085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-86258eac-2e8d-48fd-8d09-a69263c52f25 None None] MFA Rules not processed for user `ea2f6b804e164dbcb4f67926f23fdd59`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:28.907621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b177f-79a8-41cc-b278-92214f341c1e service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:28.920935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-86258eac-2e8d-48fd-8d09-a69263c52f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:28.929686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b177f-79a8-41cc-b278-92214f341c1e service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:28.946267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ea34d-c8be-4647-8f0f-9e4d76d01472 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:28.960504 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ea34d-c8be-4647-8f0f-9e4d76d01472 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:28.961001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ea34d-c8be-4647-8f0f-9e4d76d01472 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:28.961919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ea34d-c8be-4647-8f0f-9e4d76d01472 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:28.991436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ea34d-c8be-4647-8f0f-9e4d76d01472 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:28.993360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ea34d-c8be-4647-8f0f-9e4d76d01472 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:29.016624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 840/1692] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:29.019088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ea34d-c8be-4647-8f0f-9e4d76d01472 service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:29.022196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-869ea34d-c8be-4647-8f0f-9e4d76d01472 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:29.108974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 853/1693] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:29.205139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d35146-0d40-4e49-8356-4eeab28dbe1d None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:29.205605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d35146-0d40-4e49-8356-4eeab28dbe1d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:29.206032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d35146-0d40-4e49-8356-4eeab28dbe1d None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:29.207868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 854/1694] 10.222.0.59 () {54 vars in 961 bytes} [Fri Sep 16 08:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 08:05:29.214001 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef2cc0-b4d3-4c00-96a3-ed6105cdaf1c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:29.214714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef2cc0-b4d3-4c00-96a3-ed6105cdaf1c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:29.215146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef2cc0-b4d3-4c00-96a3-ed6105cdaf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:29.227136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ea34d-c8be-4647-8f0f-9e4d76d01472 service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:29.234449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee9b39-91a9-4637-9052-8bd3682461bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:29.235249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee9b39-91a9-4637-9052-8bd3682461bf service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:29.235650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee9b39-91a9-4637-9052-8bd3682461bf service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:29.239392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee9b39-91a9-4637-9052-8bd3682461bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:29.247767 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d3ef2cc0-b4d3-4c00-96a3-ed6105cdaf1c None None] MFA Rules not processed for user `c2dca05dfe8b4af0a02ea93fbe29c556`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:29.249381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d3ef2cc0-b4d3-4c00-96a3-ed6105cdaf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:29.266988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-13ee9b39-91a9-4637-9052-8bd3682461bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:29.269433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 855/1695] 10.222.0.59 () {58 vars in 1058 bytes} [Fri Sep 16 08:05:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:29.277616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef2cc0-b4d3-4c00-96a3-ed6105cdaf1c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:29.296781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef2cc0-b4d3-4c00-96a3-ed6105cdaf1c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:29.306620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0ab54-b80e-4625-ab28-289ca00fcf0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:29.307110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee9b39-91a9-4637-9052-8bd3682461bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:29.307344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0ab54-b80e-4625-ab28-289ca00fcf0c service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:29.307612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0ab54-b80e-4625-ab28-289ca00fcf0c service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:29.307890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0ab54-b80e-4625-ab28-289ca00fcf0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:29.308796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee9b39-91a9-4637-9052-8bd3682461bf service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:29.319562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:29.326286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0ab54-b80e-4625-ab28-289ca00fcf0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:29.328370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0ab54-b80e-4625-ab28-289ca00fcf0c service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:29.341450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 856/1696] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:29.359414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 841/1697] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:29.492863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14740626-5d22-4f7f-9be9-53147413daf8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:29.493184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14740626-5d22-4f7f-9be9-53147413daf8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:29.493435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14740626-5d22-4f7f-9be9-53147413daf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:29.525621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-14740626-5d22-4f7f-9be9-53147413daf8 None None] MFA Rules not processed for user `5a85cff936184b5d94375fd103493950`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:29.527742 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-14740626-5d22-4f7f-9be9-53147413daf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:29.536801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 857/1698] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:30.131866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee9b39-91a9-4637-9052-8bd3682461bf service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:30.131866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-13ee9b39-91a9-4637-9052-8bd3682461bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:30.281215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:30.304144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee9b39-91a9-4637-9052-8bd3682461bf service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:30.331170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58c60c-6556-46a7-a596-5bfddfbede71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:30.341635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58c60c-6556-46a7-a596-5bfddfbede71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:30.342265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58c60c-6556-46a7-a596-5bfddfbede71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:30.343315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58c60c-6556-46a7-a596-5bfddfbede71 admin admin] PATH_INFO: `/v3/users/755487a8ea3d48a58983efea5c7a1d62` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:30.347742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58c60c-6556-46a7-a596-5bfddfbede71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755487a8ea3d48a58983efea5c7a1d62)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:30.355945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58c60c-6556-46a7-a596-5bfddfbede71 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:30.614119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1f58c60c-6556-46a7-a596-5bfddfbede71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755487a8ea3d48a58983efea5c7a1d62'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:30.685300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1f58c60c-6556-46a7-a596-5bfddfbede71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755487a8ea3d48a58983efea5c7a1d62'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:30.718660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1f58c60c-6556-46a7-a596-5bfddfbede71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755487a8ea3d48a58983efea5c7a1d62'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:30.779926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14740626-5d22-4f7f-9be9-53147413daf8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:30.798359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14740626-5d22-4f7f-9be9-53147413daf8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:30.810584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:30.811399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:30.811680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:30.811969 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] PATH_INFO: `/v3/users/0a82437ccb64457b945085c3b57c7a9a` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:30.814081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a82437ccb64457b945085c3b57c7a9a)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:30.819265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:30.850147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1f58c60c-6556-46a7-a596-5bfddfbede71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755487a8ea3d48a58983efea5c7a1d62'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:30.850826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1f58c60c-6556-46a7-a596-5bfddfbede71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755487a8ea3d48a58983efea5c7a1d62'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:30.859643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 842/1699] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:30 2022] DELETE /identity/v3/users/755487a8ea3d48a58983efea5c7a1d62 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:30.864795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a79e12-4a22-4a76-9630-4353982ce6be None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:30.864795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a79e12-4a22-4a76-9630-4353982ce6be None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:30.865049 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a79e12-4a22-4a76-9630-4353982ce6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:30.913453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-03a79e12-4a22-4a76-9630-4353982ce6be None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:30.915510 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-03a79e12-4a22-4a76-9630-4353982ce6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:31.097957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 843/1700] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:31.109638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a79e12-4a22-4a76-9630-4353982ce6be None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:31.111750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-03a79e12-4a22-4a76-9630-4353982ce6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:31.198467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a79e12-4a22-4a76-9630-4353982ce6be None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:31.207072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca85841-65e1-421a-a358-a519c7f07916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:31.207709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85841-65e1-421a-a358-a519c7f07916 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:31.207986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85841-65e1-421a-a358-a519c7f07916 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:31.208257 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85841-65e1-421a-a358-a519c7f07916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:31.221425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca85841-65e1-421a-a358-a519c7f07916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:31.225361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca85841-65e1-421a-a358-a519c7f07916 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:31.238281 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 844/1701] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:05:31.279661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a82437ccb64457b945085c3b57c7a9a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:31.308624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a82437ccb64457b945085c3b57c7a9a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:31.327343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a82437ccb64457b945085c3b57c7a9a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:31.389950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a82437ccb64457b945085c3b57c7a9a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:31.390520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a82437ccb64457b945085c3b57c7a9a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:31.406017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 858/1702] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:30 2022] DELETE /identity/v3/users/0a82437ccb64457b945085c3b57c7a9a => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:31.406334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d50a6e-99f5-448c-b873-c104b7e37ba5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:31.406819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d50a6e-99f5-448c-b873-c104b7e37ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:31.406819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d50a6e-99f5-448c-b873-c104b7e37ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:31.408384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:31.414398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:31.479176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b6d50a6e-99f5-448c-b873-c104b7e37ba5 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:31.488191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b6d50a6e-99f5-448c-b873-c104b7e37ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:31.536294 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:31.757846 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 845/1703] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:31.794722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6ceaa-4dca-4501-8c77-44e6714641ca admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:31.815181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:31.817604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:31.818094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:31.819034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] PATH_INFO: `/v3/users/b5895120931b431884b32ce8ffb704af` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:31.824016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5895120931b431884b32ce8ffb704af)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:31.827996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:31.894129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d50a6e-99f5-448c-b873-c104b7e37ba5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:31.895949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b6d50a6e-99f5-448c-b873-c104b7e37ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:32.156855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d50a6e-99f5-448c-b873-c104b7e37ba5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:32.162092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8215759-02d4-4b24-b12a-6fc6c829a467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:32.165286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8215759-02d4-4b24-b12a-6fc6c829a467 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:32.165286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8215759-02d4-4b24-b12a-6fc6c829a467 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:32.165286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8215759-02d4-4b24-b12a-6fc6c829a467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:32.208379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5895120931b431884b32ce8ffb704af'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:32.217329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8215759-02d4-4b24-b12a-6fc6c829a467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:32.222666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8215759-02d4-4b24-b12a-6fc6c829a467 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:32.242662 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:32.247332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5895120931b431884b32ce8ffb704af'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:32.264773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 846/1704] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:05:32.265763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5895120931b431884b32ce8ffb704af'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:32.363240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5895120931b431884b32ce8ffb704af'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:32.364012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5895120931b431884b32ce8ffb704af'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:32.390456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 859/1705] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:31 2022] DELETE /identity/v3/users/b5895120931b431884b32ce8ffb704af => generated 0 bytes in 981 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:32.392516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de551215-dddf-4de3-b918-d00b902edb09 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:32.393031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de551215-dddf-4de3-b918-d00b902edb09 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:32.393455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de551215-dddf-4de3-b918-d00b902edb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:32.435723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-de551215-dddf-4de3-b918-d00b902edb09 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:32.437135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-de551215-dddf-4de3-b918-d00b902edb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:32.581496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 847/1706] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:32.593945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:32.595554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:32.633160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de551215-dddf-4de3-b918-d00b902edb09 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:32.634810 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-de551215-dddf-4de3-b918-d00b902edb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:32.698608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de551215-dddf-4de3-b918-d00b902edb09 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:32.713232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:32.715175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:32.715465 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:32.723829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] PATH_INFO: `/v3/projects/ac6baea7843b47109de92534d1934eed` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:32.723829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac6baea7843b47109de92534d1934eed)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:32.723829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:32.736239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3733b-04fe-4fba-a4a8-d535b79f4ce5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:32.750361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44179ea-789c-4e35-9fdd-8634375540bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:32.751475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44179ea-789c-4e35-9fdd-8634375540bd service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:32.751971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44179ea-789c-4e35-9fdd-8634375540bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:32.752480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44179ea-789c-4e35-9fdd-8634375540bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:32.784042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44179ea-789c-4e35-9fdd-8634375540bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:32.786225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44179ea-789c-4e35-9fdd-8634375540bd service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:32.798324 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 860/1707] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:32.933447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac6baea7843b47109de92534d1934eed', 'request_id': 'req-476538fa-af24-4289-9c8c-2df815c7b9a2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:32.948552 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac6baea7843b47109de92534d1934eed', 'request_id': 'req-476538fa-af24-4289-9c8c-2df815c7b9a2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:32.964162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac6baea7843b47109de92534d1934eed', 'request_id': 'req-476538fa-af24-4289-9c8c-2df815c7b9a2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:33.130263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] The token cache is being invalidate because project ac6baea7843b47109de92534d1934eed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:33.130577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:33.132527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-476538fa-af24-4289-9c8c-2df815c7b9a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:33.138245 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 848/1708] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:32 2022] DELETE /identity/v3/projects/ac6baea7843b47109de92534d1934eed => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:33.152934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a587c-8041-4a6f-b99c-add299580708 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:33.153914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a587c-8041-4a6f-b99c-add299580708 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:33.156403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a587c-8041-4a6f-b99c-add299580708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:33.204842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-fb1a587c-8041-4a6f-b99c-add299580708 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:33.206251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fb1a587c-8041-4a6f-b99c-add299580708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:33.376975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:33.429401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 861/1709] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:33.437171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a444e-1db5-490e-b531-118075084b44 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:33.437686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a444e-1db5-490e-b531-118075084b44 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:33.437773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a444e-1db5-490e-b531-118075084b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:33.488533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-023a444e-1db5-490e-b531-118075084b44 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:33.491815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-023a444e-1db5-490e-b531-118075084b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:33.536723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 849/1710] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:33.540818 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a587c-8041-4a6f-b99c-add299580708 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:33.563988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a587c-8041-4a6f-b99c-add299580708 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:33.582627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be1d62-d45d-4447-8ffc-531afdd631a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:33.583536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be1d62-d45d-4447-8ffc-531afdd631a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:33.585582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be1d62-d45d-4447-8ffc-531afdd631a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:33.586034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be1d62-d45d-4447-8ffc-531afdd631a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:33.587628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be1d62-d45d-4447-8ffc-531afdd631a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:33.593589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be1d62-d45d-4447-8ffc-531afdd631a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:33.595489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be1d62-d45d-4447-8ffc-531afdd631a5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:33.629127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 862/1711] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:33.634414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a444e-1db5-490e-b531-118075084b44 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:33.650374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a444e-1db5-490e-b531-118075084b44 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:33.670579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7c1ca-a228-4963-907c-6b81715af6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:33.671629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7c1ca-a228-4963-907c-6b81715af6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:33.672052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7c1ca-a228-4963-907c-6b81715af6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:33.672574 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7c1ca-a228-4963-907c-6b81715af6c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:33.676283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7c1ca-a228-4963-907c-6b81715af6c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:33.679602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7c1ca-a228-4963-907c-6b81715af6c2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:33.863158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 850/1712] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:33 2022] POST /identity/v3/projects => generated 363 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:33.868030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be1d62-d45d-4447-8ffc-531afdd631a5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:33.883217 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be1d62-d45d-4447-8ffc-531afdd631a5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:33.912666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7c1ca-a228-4963-907c-6b81715af6c2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:33.914831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-08e7c1ca-a228-4963-907c-6b81715af6c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:34.071211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf410b-e6e9-4ef5-81b6-bf5c747d3bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:34.071895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf410b-e6e9-4ef5-81b6-bf5c747d3bca admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:34.072231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf410b-e6e9-4ef5-81b6-bf5c747d3bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:34.072509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf410b-e6e9-4ef5-81b6-bf5c747d3bca admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:34.073166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf410b-e6e9-4ef5-81b6-bf5c747d3bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:34.075192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf410b-e6e9-4ef5-81b6-bf5c747d3bca admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:34.141073 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7c1ca-a228-4963-907c-6b81715af6c2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:34.176583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:34.177500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:34.177998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:34.178483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] PATH_INFO: `/v3/projects/818276d7e2d74ad6b1d24b201f75acb2` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:34.179878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=818276d7e2d74ad6b1d24b201f75acb2)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:34.183239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:34.294344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 863/1713] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:33 2022] POST /identity/v3/users => generated 282 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:05:34.296411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf410b-e6e9-4ef5-81b6-bf5c747d3bca admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:34.318193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cbaf410b-e6e9-4ef5-81b6-bf5c747d3bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:34.358655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf410b-e6e9-4ef5-81b6-bf5c747d3bca admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:34.389224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:34.393202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:34.394014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:34.394500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] PATH_INFO: `/v3/projects/8bd2d82a53b14363a6c82a6d4f2927e2` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:34.398707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bd2d82a53b14363a6c82a6d4f2927e2)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:34.401595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:34.404311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '818276d7e2d74ad6b1d24b201f75acb2', 'request_id': 'req-48452934-ceb0-4da0-8636-a9eb51def3fd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:34.442942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '818276d7e2d74ad6b1d24b201f75acb2', 'request_id': 'req-48452934-ceb0-4da0-8636-a9eb51def3fd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:34.462015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '818276d7e2d74ad6b1d24b201f75acb2', 'request_id': 'req-48452934-ceb0-4da0-8636-a9eb51def3fd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:34.471516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:34.620383 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bd2d82a53b14363a6c82a6d4f2927e2', 'request_id': 'req-44adc192-9ccf-418a-ab6a-cfa26c205f78'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:34.634246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bd2d82a53b14363a6c82a6d4f2927e2', 'request_id': 'req-44adc192-9ccf-418a-ab6a-cfa26c205f78'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:34.646039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] The token cache is being invalidate because project 818276d7e2d74ad6b1d24b201f75acb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:34.646634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:34.647901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:34.651651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 851/1714] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:33 2022] DELETE /identity/v3/projects/818276d7e2d74ad6b1d24b201f75acb2 => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:34.653358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:34.657394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:34.667847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bd2d82a53b14363a6c82a6d4f2927e2', 'request_id': 'req-44adc192-9ccf-418a-ab6a-cfa26c205f78'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:34.782491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48452934-ceb0-4da0-8636-a9eb51def3fd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:34.786624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] The token cache is being invalidate because project 8bd2d82a53b14363a6c82a6d4f2927e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:34.788777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:34.791199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:34.795765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 864/1715] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:34 2022] DELETE /identity/v3/projects/8bd2d82a53b14363a6c82a6d4f2927e2 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:34.800038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:34.805763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:34.933886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fd9db-4f42-4cba-965a-f4189b405850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:34.935204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fd9db-4f42-4cba-965a-f4189b405850 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:34.935495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fd9db-4f42-4cba-965a-f4189b405850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:34.935768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fd9db-4f42-4cba-965a-f4189b405850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:34.936439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fd9db-4f42-4cba-965a-f4189b405850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:34.940558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fd9db-4f42-4cba-965a-f4189b405850 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:34.942818 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc192-9ccf-418a-ab6a-cfa26c205f78 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:34.957380 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f362bd6-afd6-4a64-93fe-4a0dcf65bd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:34.962112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f362bd6-afd6-4a64-93fe-4a0dcf65bd1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:34.962112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f362bd6-afd6-4a64-93fe-4a0dcf65bd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:34.962112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f362bd6-afd6-4a64-93fe-4a0dcf65bd1a admin admin] PATH_INFO: `/v3/users/0304c4e7f6b34b44b3fcbdca0d8e9b6e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:34.963291 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f362bd6-afd6-4a64-93fe-4a0dcf65bd1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0304c4e7f6b34b44b3fcbdca0d8e9b6e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:34.966971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f362bd6-afd6-4a64-93fe-4a0dcf65bd1a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:34.989180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 852/1716] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:34.989180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fd9db-4f42-4cba-965a-f4189b405850 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:34.996279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-249fd9db-4f42-4cba-965a-f4189b405850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:35.058052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fd9db-4f42-4cba-965a-f4189b405850 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:35.073317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:35.074161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:35.075485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:35.075894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] PATH_INFO: `/v3/users/d2db04b6b2d0456bb7518a66fa631963` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:35.077284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2db04b6b2d0456bb7518a66fa631963)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:35.080236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:35.296946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5f362bd6-afd6-4a64-93fe-4a0dcf65bd1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0304c4e7f6b34b44b3fcbdca0d8e9b6e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:35.327121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5f362bd6-afd6-4a64-93fe-4a0dcf65bd1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0304c4e7f6b34b44b3fcbdca0d8e9b6e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:35.341703 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5f362bd6-afd6-4a64-93fe-4a0dcf65bd1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0304c4e7f6b34b44b3fcbdca0d8e9b6e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:35.371456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:35.397158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2db04b6b2d0456bb7518a66fa631963'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:35.413734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2db04b6b2d0456bb7518a66fa631963'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:35.422923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2db04b6b2d0456bb7518a66fa631963'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:35.430362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5f362bd6-afd6-4a64-93fe-4a0dcf65bd1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0304c4e7f6b34b44b3fcbdca0d8e9b6e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:35.431815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5f362bd6-afd6-4a64-93fe-4a0dcf65bd1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0304c4e7f6b34b44b3fcbdca0d8e9b6e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:35.438349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 865/1717] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:34 2022] DELETE /identity/v3/users/0304c4e7f6b34b44b3fcbdca0d8e9b6e => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:35.439699 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7fee5-9458-467a-b96b-386b10adce0b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:35.440304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7fee5-9458-467a-b96b-386b10adce0b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:35.440584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7fee5-9458-467a-b96b-386b10adce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:35.461424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0ce7fee5-9458-467a-b96b-386b10adce0b None None] MFA Rules not processed for user `393fddf9c75447fab334b84694dcf4f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:35.463325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0ce7fee5-9458-467a-b96b-386b10adce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:35.523025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2db04b6b2d0456bb7518a66fa631963'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:35.525205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2db04b6b2d0456bb7518a66fa631963'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:35.526295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 866/1718] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:35.527732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7fee5-9458-467a-b96b-386b10adce0b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:35.529654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0ce7fee5-9458-467a-b96b-386b10adce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:35.536654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 853/1719] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:34 2022] DELETE /identity/v3/users/d2db04b6b2d0456bb7518a66fa631963 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:35.699945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7fee5-9458-467a-b96b-386b10adce0b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:35.819389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39672780-581e-4bd3-be63-ca28270c2891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:35.820058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39672780-581e-4bd3-be63-ca28270c2891 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:35.820728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39672780-581e-4bd3-be63-ca28270c2891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:35.820728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39672780-581e-4bd3-be63-ca28270c2891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:35.821255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39672780-581e-4bd3-be63-ca28270c2891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:35.824604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39672780-581e-4bd3-be63-ca28270c2891 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:35.853378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 867/1720] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:35.859869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:35.863262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:35.908773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e463457-bf8e-4d36-bca4-09efe816f1de admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:35.917147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f828f04-c550-498f-bbd0-61466b82d40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:35.917749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f828f04-c550-498f-bbd0-61466b82d40b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:35.918034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f828f04-c550-498f-bbd0-61466b82d40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:35.918541 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f828f04-c550-498f-bbd0-61466b82d40b admin admin] PATH_INFO: `/v3/projects/633fb3e0018848769357421eeb26ba46/users/e0a14fb35a064394b0a276eb25409fb6/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:35.940781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f828f04-c550-498f-bbd0-61466b82d40b admin admin] RBAC: Authorizing `identity:create_grant(project_id=633fb3e0018848769357421eeb26ba46, user_id=e0a14fb35a064394b0a276eb25409fb6, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:35.943669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f828f04-c550-498f-bbd0-61466b82d40b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:35.993783 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9ea5f-3c1e-4a57-b619-6bd7bf8d29a5 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:35.994323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9ea5f-3c1e-4a57-b619-6bd7bf8d29a5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:35.994323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9ea5f-3c1e-4a57-b619-6bd7bf8d29a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:36.070431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 854/1721] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:35 2022] PUT /identity/v3/projects/633fb3e0018848769357421eeb26ba46/users/e0a14fb35a064394b0a276eb25409fb6/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:36.089684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8736-5288-46ca-ba3a-503b938674f2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:36.090202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8736-5288-46ca-ba3a-503b938674f2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:36.090644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8736-5288-46ca-ba3a-503b938674f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:36.100670 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-edd9ea5f-3c1e-4a57-b619-6bd7bf8d29a5 None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:36.105841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-edd9ea5f-3c1e-4a57-b619-6bd7bf8d29a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:36.155558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4e7e8736-5288-46ca-ba3a-503b938674f2 None None] MFA Rules not processed for user `e0a14fb35a064394b0a276eb25409fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:36.160092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4e7e8736-5288-46ca-ba3a-503b938674f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:36.287728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 868/1722] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:05:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:36.428899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 855/1723] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:36.433577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9ea5f-3c1e-4a57-b619-6bd7bf8d29a5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:36.556170 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9ea5f-3c1e-4a57-b619-6bd7bf8d29a5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:36.561195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f798f-7cf5-4e1b-91a9-79d6a0e82f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:36.561832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f798f-7cf5-4e1b-91a9-79d6a0e82f51 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:36.562118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f798f-7cf5-4e1b-91a9-79d6a0e82f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:36.562448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f798f-7cf5-4e1b-91a9-79d6a0e82f51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:36.563857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f798f-7cf5-4e1b-91a9-79d6a0e82f51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:36.566039 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f798f-7cf5-4e1b-91a9-79d6a0e82f51 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:36.754730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 869/1724] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:36 2022] POST /identity/v3/projects => generated 363 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:36.759494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e8736-5288-46ca-ba3a-503b938674f2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:36.774989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e8736-5288-46ca-ba3a-503b938674f2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:36.813050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f798f-7cf5-4e1b-91a9-79d6a0e82f51 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:36.815887 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-417f798f-7cf5-4e1b-91a9-79d6a0e82f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:36.919495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbc992-78a5-4247-ab6d-68fe06c474d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:36.920132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbc992-78a5-4247-ab6d-68fe06c474d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:36.920422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbc992-78a5-4247-ab6d-68fe06c474d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:36.920700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbc992-78a5-4247-ab6d-68fe06c474d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:36.925929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbc992-78a5-4247-ab6d-68fe06c474d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:36.925929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbc992-78a5-4247-ab6d-68fe06c474d4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:37.052652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 856/1725] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:36 2022] POST /identity/v3/users => generated 288 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:37.056127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbc992-78a5-4247-ab6d-68fe06c474d4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:37.081126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbc992-78a5-4247-ab6d-68fe06c474d4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:37.096406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f798f-7cf5-4e1b-91a9-79d6a0e82f51 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:37.101503 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c9854-8c7d-4a8a-b956-ee1b1fbf93e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:37.102267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c9854-8c7d-4a8a-b956-ee1b1fbf93e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:37.102623 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c9854-8c7d-4a8a-b956-ee1b1fbf93e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:37.103011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c9854-8c7d-4a8a-b956-ee1b1fbf93e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:37.103785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c9854-8c7d-4a8a-b956-ee1b1fbf93e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:37.105863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c9854-8c7d-4a8a-b956-ee1b1fbf93e7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:37.106612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af393d-a8ee-48e6-a29d-2043b2d75142 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:37.109629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af393d-a8ee-48e6-a29d-2043b2d75142 service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:37.113783 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af393d-a8ee-48e6-a29d-2043b2d75142 service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:37.114814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af393d-a8ee-48e6-a29d-2043b2d75142 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:37.121400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 857/1726] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:37.125582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c9854-8c7d-4a8a-b956-ee1b1fbf93e7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:37.136265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c9854-8c7d-4a8a-b956-ee1b1fbf93e7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:37.142873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b4a92-d61b-479d-8df6-f32349fea092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:37.142873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b4a92-d61b-479d-8df6-f32349fea092 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:37.143311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b4a92-d61b-479d-8df6-f32349fea092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:37.143705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b4a92-d61b-479d-8df6-f32349fea092 admin admin] PATH_INFO: `/v3/projects/ef38ee76551d4e8a86917c86c29c8c7e/users/026821febc8c42d8b02c08c03678e1b6/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:37.163777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b4a92-d61b-479d-8df6-f32349fea092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef38ee76551d4e8a86917c86c29c8c7e, user_id=026821febc8c42d8b02c08c03678e1b6, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:37.166624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b4a92-d61b-479d-8df6-f32349fea092 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:37.194655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af393d-a8ee-48e6-a29d-2043b2d75142 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:37.197440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af393d-a8ee-48e6-a29d-2043b2d75142 service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:37.208868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 870/1727] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:05:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:05:37.263193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 858/1728] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:37 2022] PUT /identity/v3/projects/ef38ee76551d4e8a86917c86c29c8c7e/users/026821febc8c42d8b02c08c03678e1b6/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:37.263349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af393d-a8ee-48e6-a29d-2043b2d75142 service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:37.282279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af393d-a8ee-48e6-a29d-2043b2d75142 service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:37.364417 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b4a92-d61b-479d-8df6-f32349fea092 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:37.376859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-598b4a92-d61b-479d-8df6-f32349fea092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:37.517307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:37.596726 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4647fb-6148-4b13-bb3c-fba254be5250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:37.598755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4647fb-6148-4b13-bb3c-fba254be5250 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:37.603912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4647fb-6148-4b13-bb3c-fba254be5250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:37.604178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4647fb-6148-4b13-bb3c-fba254be5250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:37.604995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4647fb-6148-4b13-bb3c-fba254be5250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:37.621522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4647fb-6148-4b13-bb3c-fba254be5250 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:37.707078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 871/1729] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:37.709378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf4f40-e7b3-4211-8fea-1a9040c5411b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:37.709508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf4f40-e7b3-4211-8fea-1a9040c5411b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:37.714759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf4f40-e7b3-4211-8fea-1a9040c5411b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:37.732443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b4a92-d61b-479d-8df6-f32349fea092 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:37.748776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:37.749030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:37.749292 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:37.755114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] PATH_INFO: `/v3/projects/eb52d54cfb144457a71c78339f74d936` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:37.760883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb52d54cfb144457a71c78339f74d936)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:37.765393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:37.799968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cfcf4f40-e7b3-4211-8fea-1a9040c5411b None None] MFA Rules not processed for user `a29c4e53ac9a438e9701a70f3de3f32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:37.804248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cfcf4f40-e7b3-4211-8fea-1a9040c5411b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:38.148670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb52d54cfb144457a71c78339f74d936', 'request_id': 'req-e4f171ae-9bbb-4053-bafd-cb01e89da878'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:38.188001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb52d54cfb144457a71c78339f74d936', 'request_id': 'req-e4f171ae-9bbb-4053-bafd-cb01e89da878'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:38.190255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 872/1730] 10.222.0.42 () {58 vars in 1015 bytes} [Fri Sep 16 08:05:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:38.191136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf4f40-e7b3-4211-8fea-1a9040c5411b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:38.227522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf4f40-e7b3-4211-8fea-1a9040c5411b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:38.233099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb52d54cfb144457a71c78339f74d936', 'request_id': 'req-e4f171ae-9bbb-4053-bafd-cb01e89da878'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:38.345592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] The token cache is being invalidate because project eb52d54cfb144457a71c78339f74d936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:38.346038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:38.347429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:38.351327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 859/1731] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:37 2022] DELETE /identity/v3/projects/eb52d54cfb144457a71c78339f74d936 => generated 0 bytes in 987 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:05:38.352780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:38.356041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:38.455390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d585c31-099d-4db6-a8d0-864125cac24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:38.459159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d585c31-099d-4db6-a8d0-864125cac24a admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:38.459159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d585c31-099d-4db6-a8d0-864125cac24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:38.459159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d585c31-099d-4db6-a8d0-864125cac24a admin admin] PATH_INFO: `/v3/projects/ef38ee76551d4e8a86917c86c29c8c7e/users/026821febc8c42d8b02c08c03678e1b6/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:38.464526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d585c31-099d-4db6-a8d0-864125cac24a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef38ee76551d4e8a86917c86c29c8c7e, user_id=026821febc8c42d8b02c08c03678e1b6, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:38.469467 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d585c31-099d-4db6-a8d0-864125cac24a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:38.501978 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:38.563328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 873/1732] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:38 2022] PUT /identity/v3/projects/ef38ee76551d4e8a86917c86c29c8c7e/users/026821febc8c42d8b02c08c03678e1b6/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:38.563328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d585c31-099d-4db6-a8d0-864125cac24a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:38.581264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2d585c31-099d-4db6-a8d0-864125cac24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:38.668684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f171ae-9bbb-4053-bafd-cb01e89da878 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:38.733852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d585c31-099d-4db6-a8d0-864125cac24a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:38.753570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb922429-03e9-46c1-921d-c3f0867f20ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:38.754434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb922429-03e9-46c1-921d-c3f0867f20ee service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:38.754888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb922429-03e9-46c1-921d-c3f0867f20ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:38.755359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb922429-03e9-46c1-921d-c3f0867f20ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:38.769120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fb922429-03e9-46c1-921d-c3f0867f20ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:38.810852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb922429-03e9-46c1-921d-c3f0867f20ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:38.812566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb922429-03e9-46c1-921d-c3f0867f20ee service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:38.836206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 874/1733] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:38.837697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26e92c-4442-46a9-ae1a-5a702f2f0e86 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:38.842384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26e92c-4442-46a9-ae1a-5a702f2f0e86 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:38.842999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26e92c-4442-46a9-ae1a-5a702f2f0e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:38.871148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:38.872514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:38.875570 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:38.876019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] PATH_INFO: `/v3/projects/2df3d15d614f4de3b5ad652b8df17e2c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:38.882027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df3d15d614f4de3b5ad652b8df17e2c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:38.884121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:38.902271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3a26e92c-4442-46a9-ae1a-5a702f2f0e86 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:38.910009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3a26e92c-4442-46a9-ae1a-5a702f2f0e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:38.970689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 875/1734] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:38.979350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5effb6-d9b9-41e5-83c1-7c497e225ae0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:38.979640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5effb6-d9b9-41e5-83c1-7c497e225ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:38.982791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5effb6-d9b9-41e5-83c1-7c497e225ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:39.043758 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5c5effb6-d9b9-41e5-83c1-7c497e225ae0 None None] MFA Rules not processed for user `026821febc8c42d8b02c08c03678e1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:39.045686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5c5effb6-d9b9-41e5-83c1-7c497e225ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:39.166650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df3d15d614f4de3b5ad652b8df17e2c', 'request_id': 'req-db71e4ed-6e84-46ae-a4c8-c6940262275d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:39.182966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df3d15d614f4de3b5ad652b8df17e2c', 'request_id': 'req-db71e4ed-6e84-46ae-a4c8-c6940262275d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:39.194035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df3d15d614f4de3b5ad652b8df17e2c', 'request_id': 'req-db71e4ed-6e84-46ae-a4c8-c6940262275d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:39.268961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 876/1735] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:39.270278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e148dc-1cd2-4485-916d-fe75a0c25ac3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:39.270569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e148dc-1cd2-4485-916d-fe75a0c25ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:39.270877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e148dc-1cd2-4485-916d-fe75a0c25ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:39.318261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] The token cache is being invalidate because project 2df3d15d614f4de3b5ad652b8df17e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:39.321221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:39.323056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:39.337085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-15e148dc-1cd2-4485-916d-fe75a0c25ac3 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:39.337276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-15e148dc-1cd2-4485-916d-fe75a0c25ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:39.338358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 860/1736] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:38 2022] DELETE /identity/v3/projects/2df3d15d614f4de3b5ad652b8df17e2c => generated 0 bytes in 982 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:39.340815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:39.348966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:39.507818 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 877/1737] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:39.509970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e127496-ed4f-4e89-a420-732b344a5a72 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:39.512035 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e127496-ed4f-4e89-a420-732b344a5a72 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:39.512432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e127496-ed4f-4e89-a420-732b344a5a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:39.516364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71e4ed-6e84-46ae-a4c8-c6940262275d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:39.527846 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6653c3-e7ce-49cd-8253-5f96c0b55b12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:39.530379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6653c3-e7ce-49cd-8253-5f96c0b55b12 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:39.531280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6653c3-e7ce-49cd-8253-5f96c0b55b12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:39.532392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6653c3-e7ce-49cd-8253-5f96c0b55b12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:39.541267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8e6653c3-e7ce-49cd-8253-5f96c0b55b12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:39.575953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9e127496-ed4f-4e89-a420-732b344a5a72 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:39.577480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9e127496-ed4f-4e89-a420-732b344a5a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:39.624069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:39.624069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 878/1738] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:39.624069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e127496-ed4f-4e89-a420-732b344a5a72 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:39.663710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e127496-ed4f-4e89-a420-732b344a5a72 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:39.678328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f776bbc-63b2-4bc5-b1b3-179f2428a037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:39.679444 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776bbc-63b2-4bc5-b1b3-179f2428a037 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:39.679992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776bbc-63b2-4bc5-b1b3-179f2428a037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:39.680286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776bbc-63b2-4bc5-b1b3-179f2428a037 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:39.680977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f776bbc-63b2-4bc5-b1b3-179f2428a037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:39.681303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f776bbc-63b2-4bc5-b1b3-179f2428a037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:39.685736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f776bbc-63b2-4bc5-b1b3-179f2428a037 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:39.715181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 879/1739] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:39.717761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800deb3b-6504-4e51-bc55-7ccd703369dc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:39.719120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800deb3b-6504-4e51-bc55-7ccd703369dc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:39.719818 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800deb3b-6504-4e51-bc55-7ccd703369dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:39.758937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-800deb3b-6504-4e51-bc55-7ccd703369dc None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:39.760201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-800deb3b-6504-4e51-bc55-7ccd703369dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:39.808523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6653c3-e7ce-49cd-8253-5f96c0b55b12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:39.810091 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 880/1740] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:39.813706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6653c3-e7ce-49cd-8253-5f96c0b55b12 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:39.815222 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800deb3b-6504-4e51-bc55-7ccd703369dc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:39.848875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 861/1741] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:39.952017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800deb3b-6504-4e51-bc55-7ccd703369dc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:39.952725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6653c3-e7ce-49cd-8253-5f96c0b55b12 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:39.953280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c75944-8f17-4409-9f57-c1a840693dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:39.953280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c75944-8f17-4409-9f57-c1a840693dfa admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:39.953280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c75944-8f17-4409-9f57-c1a840693dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:39.953280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c75944-8f17-4409-9f57-c1a840693dfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:39.953602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c75944-8f17-4409-9f57-c1a840693dfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:39.953602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c75944-8f17-4409-9f57-c1a840693dfa admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:39.953912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6653c3-e7ce-49cd-8253-5f96c0b55b12 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:39.953912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469d756-aecd-4fd2-85e5-1e959e865d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:39.953912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d756-aecd-4fd2-85e5-1e959e865d6c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:39.953912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d756-aecd-4fd2-85e5-1e959e865d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:39.954246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d756-aecd-4fd2-85e5-1e959e865d6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:39.954246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469d756-aecd-4fd2-85e5-1e959e865d6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:39.954246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469d756-aecd-4fd2-85e5-1e959e865d6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:39.954246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469d756-aecd-4fd2-85e5-1e959e865d6c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:39.976829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 862/1742] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:39.980669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469d756-aecd-4fd2-85e5-1e959e865d6c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:40.003172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469d756-aecd-4fd2-85e5-1e959e865d6c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:40.009845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec718dcb-1a80-4b19-812f-bdd37a3320ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:40.010701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec718dcb-1a80-4b19-812f-bdd37a3320ca admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:40.011137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec718dcb-1a80-4b19-812f-bdd37a3320ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:40.011557 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec718dcb-1a80-4b19-812f-bdd37a3320ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:40.012352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec718dcb-1a80-4b19-812f-bdd37a3320ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:40.014937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec718dcb-1a80-4b19-812f-bdd37a3320ca admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:40.765748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 881/1743] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:39 2022] POST /identity/v3/projects => generated 363 bytes in 955 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:40.766694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c75944-8f17-4409-9f57-c1a840693dfa admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:40.866298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 863/1744] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:39 2022] POST /identity/v3/projects => generated 387 bytes in 886 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:40.870362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec718dcb-1a80-4b19-812f-bdd37a3320ca admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:40.927099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c75944-8f17-4409-9f57-c1a840693dfa admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:40.984691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:40.988174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec718dcb-1a80-4b19-812f-bdd37a3320ca admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:40.995534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedee9ab-01f4-4f67-8a62-c77cfb15622b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:40.996491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedee9ab-01f4-4f67-8a62-c77cfb15622b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:40.997004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedee9ab-01f4-4f67-8a62-c77cfb15622b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:40.997433 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedee9ab-01f4-4f67-8a62-c77cfb15622b admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:40.998280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedee9ab-01f4-4f67-8a62-c77cfb15622b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:41.004799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedee9ab-01f4-4f67-8a62-c77cfb15622b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:41.041872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2be64-062d-4d62-aa0e-4fcca39ecca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:41.042821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2be64-062d-4d62-aa0e-4fcca39ecca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:41.043261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2be64-062d-4d62-aa0e-4fcca39ecca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:41.043726 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2be64-062d-4d62-aa0e-4fcca39ecca7 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:41.044889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2be64-062d-4d62-aa0e-4fcca39ecca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:41.047445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2be64-062d-4d62-aa0e-4fcca39ecca7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:41.118269 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 864/1745] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:40 2022] POST /identity/v3/users => generated 294 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:05:41.122719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedee9ab-01f4-4f67-8a62-c77cfb15622b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:41.140580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedee9ab-01f4-4f67-8a62-c77cfb15622b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:41.147143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b8a04-28c3-498f-981c-336f85d45cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:41.147955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b8a04-28c3-498f-981c-336f85d45cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:41.148851 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b8a04-28c3-498f-981c-336f85d45cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:41.149261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b8a04-28c3-498f-981c-336f85d45cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:41.150063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b8a04-28c3-498f-981c-336f85d45cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:41.152350 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b8a04-28c3-498f-981c-336f85d45cf0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:41.167251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 865/1746] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:05:41.171111 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b8a04-28c3-498f-981c-336f85d45cf0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:41.180534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 882/1747] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:40 2022] POST /identity/v3/users => generated 282 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:41.181790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2be64-062d-4d62-aa0e-4fcca39ecca7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:41.183678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b8a04-28c3-498f-981c-336f85d45cf0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:41.188797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b76d3-e0af-4a7a-9431-e68c45c591c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:41.189604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b76d3-e0af-4a7a-9431-e68c45c591c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:41.190035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b76d3-e0af-4a7a-9431-e68c45c591c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:41.190456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b76d3-e0af-4a7a-9431-e68c45c591c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:41.191434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b76d3-e0af-4a7a-9431-e68c45c591c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:41.194176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b76d3-e0af-4a7a-9431-e68c45c591c0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:41.204798 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2be64-062d-4d62-aa0e-4fcca39ecca7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:41.215249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:41.224072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3809f59-e952-4b88-becc-aaefa3fe0c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:41.227076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3809f59-e952-4b88-becc-aaefa3fe0c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:41.227553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3809f59-e952-4b88-becc-aaefa3fe0c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:41.228035 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3809f59-e952-4b88-becc-aaefa3fe0c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:41.228859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3809f59-e952-4b88-becc-aaefa3fe0c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:41.231360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 866/1748] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:41.234929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3809f59-e952-4b88-becc-aaefa3fe0c2a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:41.236133 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b76d3-e0af-4a7a-9431-e68c45c591c0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:41.286459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 883/1749] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:05:41.287966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3809f59-e952-4b88-becc-aaefa3fe0c2a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:41.300811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b76d3-e0af-4a7a-9431-e68c45c591c0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:41.322108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82a691-d411-4134-9093-4c4d85a612e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:41.323399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82a691-d411-4134-9093-4c4d85a612e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:41.323772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82a691-d411-4134-9093-4c4d85a612e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:41.324136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82a691-d411-4134-9093-4c4d85a612e3 admin admin] PATH_INFO: `/v3/projects/00acb1faba2d4d28b8e72ea2c435a613/users/c2e6434ca3ce4bcaa38adae095d9c1e4/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:41.325577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3809f59-e952-4b88-becc-aaefa3fe0c2a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:41.332852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff27c4-a9dd-49a9-b7ab-01c89269868e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:41.333772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff27c4-a9dd-49a9-b7ab-01c89269868e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:41.334305 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff27c4-a9dd-49a9-b7ab-01c89269868e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:41.334819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff27c4-a9dd-49a9-b7ab-01c89269868e admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:41.335650 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff27c4-a9dd-49a9-b7ab-01c89269868e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:41.337634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff27c4-a9dd-49a9-b7ab-01c89269868e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:41.350713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 884/1750] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:41.354583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff27c4-a9dd-49a9-b7ab-01c89269868e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:41.367832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff27c4-a9dd-49a9-b7ab-01c89269868e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:41.400257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5b1cd-2f89-4c3a-8aaa-90b33fe312b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:41.401734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82a691-d411-4134-9093-4c4d85a612e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00acb1faba2d4d28b8e72ea2c435a613, user_id=c2e6434ca3ce4bcaa38adae095d9c1e4, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:41.403268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5b1cd-2f89-4c3a-8aaa-90b33fe312b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:41.404891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82a691-d411-4134-9093-4c4d85a612e3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:41.405852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5b1cd-2f89-4c3a-8aaa-90b33fe312b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:41.408043 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5b1cd-2f89-4c3a-8aaa-90b33fe312b4 admin admin] PATH_INFO: `/v3/projects/eb1e7f8f9f344c3cbc5ab982e1127153/users/9171d03e8ba94f139472d1a9c48be9d3/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:41.469384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5b1cd-2f89-4c3a-8aaa-90b33fe312b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb1e7f8f9f344c3cbc5ab982e1127153, user_id=9171d03e8ba94f139472d1a9c48be9d3, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:41.474317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5b1cd-2f89-4c3a-8aaa-90b33fe312b4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:41.543406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 867/1751] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:41 2022] PUT /identity/v3/projects/00acb1faba2d4d28b8e72ea2c435a613/users/c2e6434ca3ce4bcaa38adae095d9c1e4/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:41.555934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27524ba-4ed7-411e-b727-11888b91e5cb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:41.557144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27524ba-4ed7-411e-b727-11888b91e5cb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:41.557646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27524ba-4ed7-411e-b727-11888b91e5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:41.590736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 885/1752] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:41 2022] PUT /identity/v3/projects/eb1e7f8f9f344c3cbc5ab982e1127153/users/9171d03e8ba94f139472d1a9c48be9d3/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:41.604779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138f502-1925-46da-9a48-9d5c506c6ce2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:41.605073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138f502-1925-46da-9a48-9d5c506c6ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:41.605355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138f502-1925-46da-9a48-9d5c506c6ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:41.654215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c27524ba-4ed7-411e-b727-11888b91e5cb None None] MFA Rules not processed for user `c2e6434ca3ce4bcaa38adae095d9c1e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:41.655699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c27524ba-4ed7-411e-b727-11888b91e5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:41.714483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7138f502-1925-46da-9a48-9d5c506c6ce2 None None] MFA Rules not processed for user `9171d03e8ba94f139472d1a9c48be9d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:41.715791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7138f502-1925-46da-9a48-9d5c506c6ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:41.980683 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 868/1753] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:41.983351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27524ba-4ed7-411e-b727-11888b91e5cb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:42.046091 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 886/1754] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:42.051149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138f502-1925-46da-9a48-9d5c506c6ce2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:42.132611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27524ba-4ed7-411e-b727-11888b91e5cb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:42.139208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d297e-26e5-4f83-8bf9-34ac0c2cb9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:42.141933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d297e-26e5-4f83-8bf9-34ac0c2cb9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:42.141933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d297e-26e5-4f83-8bf9-34ac0c2cb9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:42.141933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d297e-26e5-4f83-8bf9-34ac0c2cb9e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:42.141933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d297e-26e5-4f83-8bf9-34ac0c2cb9e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:42.152802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d297e-26e5-4f83-8bf9-34ac0c2cb9e9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:42.226282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138f502-1925-46da-9a48-9d5c506c6ce2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:42.240898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4e218-e81c-473f-93a3-283f52463234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:42.242453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4e218-e81c-473f-93a3-283f52463234 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:42.243155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4e218-e81c-473f-93a3-283f52463234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:42.243584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4e218-e81c-473f-93a3-283f52463234 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:42.247944 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4e218-e81c-473f-93a3-283f52463234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:42.251379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4e218-e81c-473f-93a3-283f52463234 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:42.332121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 869/1755] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:41 2022] POST /identity/v3/projects => generated 387 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:42.332121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d297e-26e5-4f83-8bf9-34ac0c2cb9e9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:42.337130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3d4d297e-26e5-4f83-8bf9-34ac0c2cb9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:42.355686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 887/1756] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:42 2022] POST /identity/v3/projects => generated 361 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:42.359404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4e218-e81c-473f-93a3-283f52463234 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:42.517049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4e218-e81c-473f-93a3-283f52463234 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:42.525810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4deff-7e53-45f6-9046-fff6a2a0b46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:42.525810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4deff-7e53-45f6-9046-fff6a2a0b46e admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:42.525810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4deff-7e53-45f6-9046-fff6a2a0b46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:42.525810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4deff-7e53-45f6-9046-fff6a2a0b46e admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:42.526152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4deff-7e53-45f6-9046-fff6a2a0b46e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:42.528376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4deff-7e53-45f6-9046-fff6a2a0b46e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:42.554390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d297e-26e5-4f83-8bf9-34ac0c2cb9e9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:42.560182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:42.561181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:42.561549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:42.561841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] PATH_INFO: `/v3/users/e0a14fb35a064394b0a276eb25409fb6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:42.563057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0a14fb35a064394b0a276eb25409fb6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:42.565637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:42.654152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 888/1757] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:42 2022] POST /identity/v3/users => generated 300 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:42.659559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4deff-7e53-45f6-9046-fff6a2a0b46e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:42.703166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4deff-7e53-45f6-9046-fff6a2a0b46e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:42.727614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170630b-6c39-4b34-8d1f-ca512781aeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:42.730086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170630b-6c39-4b34-8d1f-ca512781aeec admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:42.730646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170630b-6c39-4b34-8d1f-ca512781aeec admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:42.732199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170630b-6c39-4b34-8d1f-ca512781aeec admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:42.732972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170630b-6c39-4b34-8d1f-ca512781aeec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:42.739793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170630b-6c39-4b34-8d1f-ca512781aeec admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:42.852883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0a14fb35a064394b0a276eb25409fb6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:42.872249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0a14fb35a064394b0a276eb25409fb6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:42.890182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0a14fb35a064394b0a276eb25409fb6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:42.894561 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:42.934368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 889/1758] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:42 2022] POST /identity/v3/users => generated 287 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:42.935950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e247ad-82f7-42f3-9007-bfb79280287c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:42.936239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e247ad-82f7-42f3-9007-bfb79280287c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:42.936884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e247ad-82f7-42f3-9007-bfb79280287c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:42.939942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 890/1759] 10.222.0.59 () {54 vars in 921 bytes} [Fri Sep 16 08:05:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:05:42.942025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e247ad-82f7-42f3-9007-bfb79280287c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:42.964863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e247ad-82f7-42f3-9007-bfb79280287c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:42.981409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0a14fb35a064394b0a276eb25409fb6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:42.982633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0a14fb35a064394b0a276eb25409fb6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:42.993409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 870/1760] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:42 2022] DELETE /identity/v3/users/e0a14fb35a064394b0a276eb25409fb6 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:42.994650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:42.997293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:43.001772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8eb16-bd4a-4044-96a8-2d605f10ed40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:43.002310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8eb16-bd4a-4044-96a8-2d605f10ed40 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:43.002589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8eb16-bd4a-4044-96a8-2d605f10ed40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:43.010908 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8eb16-bd4a-4044-96a8-2d605f10ed40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:43.013940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8eb16-bd4a-4044-96a8-2d605f10ed40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:43.019816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8eb16-bd4a-4044-96a8-2d605f10ed40 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:43.055695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 891/1761] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:43.058893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac762112-8b87-4988-9fd0-beb37d7449f8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:43.058893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac762112-8b87-4988-9fd0-beb37d7449f8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:43.058893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac762112-8b87-4988-9fd0-beb37d7449f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:43.089374 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ac762112-8b87-4988-9fd0-beb37d7449f8 None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:43.096619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ac762112-8b87-4988-9fd0-beb37d7449f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:43.218186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e571c6-9496-4a9f-a0dd-965d727c2668 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:43.237140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda1e68-e6ce-4372-ae9e-6d3d4dbfef6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:43.238016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda1e68-e6ce-4372-ae9e-6d3d4dbfef6a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:43.238445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda1e68-e6ce-4372-ae9e-6d3d4dbfef6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:43.238942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda1e68-e6ce-4372-ae9e-6d3d4dbfef6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:43.240366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda1e68-e6ce-4372-ae9e-6d3d4dbfef6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:43.242185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda1e68-e6ce-4372-ae9e-6d3d4dbfef6a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:43.266476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 871/1762] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:43.268320 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67203b91-f063-4b8c-8160-bd7c8d73a02f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:43.268889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67203b91-f063-4b8c-8160-bd7c8d73a02f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:43.269309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67203b91-f063-4b8c-8160-bd7c8d73a02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:43.294285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 892/1763] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Sep 16 08:05:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:43.295023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac762112-8b87-4988-9fd0-beb37d7449f8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:43.298504 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-67203b91-f063-4b8c-8160-bd7c8d73a02f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:43.300807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-67203b91-f063-4b8c-8160-bd7c8d73a02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:43.303105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ac762112-8b87-4988-9fd0-beb37d7449f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:43.345627 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 872/1764] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:43.347175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd8c90-bed6-46ef-81db-4e0729e417f5 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:43.347405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd8c90-bed6-46ef-81db-4e0729e417f5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:43.347686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd8c90-bed6-46ef-81db-4e0729e417f5 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:43.349013 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 873/1765] 10.222.0.59 () {54 vars in 921 bytes} [Fri Sep 16 08:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:05:43.350501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd8c90-bed6-46ef-81db-4e0729e417f5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:43.363419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac762112-8b87-4988-9fd0-beb37d7449f8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:43.373107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd8c90-bed6-46ef-81db-4e0729e417f5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:43.378618 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8ded-6793-47a4-847e-066f3435afa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:43.379495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8ded-6793-47a4-847e-066f3435afa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:43.379779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8ded-6793-47a4-847e-066f3435afa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:43.380066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8ded-6793-47a4-847e-066f3435afa2 admin admin] PATH_INFO: `/v3/projects/0e9a6bbc2cb2480986bf06cfe54476c4/users/17a62ec718b74813a71f5e74f3f4a937/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:43.390005 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634307d-3f6c-49d8-b60f-50384dccb744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:43.390489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634307d-3f6c-49d8-b60f-50384dccb744 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:43.390802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634307d-3f6c-49d8-b60f-50384dccb744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:43.391087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634307d-3f6c-49d8-b60f-50384dccb744 admin admin] PATH_INFO: `/v3/projects/e8f0589ac89845c2b026663cdb0ad4f3/users/2a26f1da89984e4cb9e361ea536ed53b/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:43.405830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a8ded-6793-47a4-847e-066f3435afa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9a6bbc2cb2480986bf06cfe54476c4, user_id=17a62ec718b74813a71f5e74f3f4a937, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:43.416456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a8ded-6793-47a4-847e-066f3435afa2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:43.447936 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634307d-3f6c-49d8-b60f-50384dccb744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f0589ac89845c2b026663cdb0ad4f3, user_id=2a26f1da89984e4cb9e361ea536ed53b, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:43.448743 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:43.452618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634307d-3f6c-49d8-b60f-50384dccb744 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:43.512017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 874/1766] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:43 2022] PUT /identity/v3/projects/0e9a6bbc2cb2480986bf06cfe54476c4/users/17a62ec718b74813a71f5e74f3f4a937/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:43.515783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8ded-6793-47a4-847e-066f3435afa2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:43.516040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0e5a8ded-6793-47a4-847e-066f3435afa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:43.543749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 893/1767] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:43 2022] PUT /identity/v3/projects/e8f0589ac89845c2b026663cdb0ad4f3/users/2a26f1da89984e4cb9e361ea536ed53b/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:43.544805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634307d-3f6c-49d8-b60f-50384dccb744 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:43.548063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f634307d-3f6c-49d8-b60f-50384dccb744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:43.680019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8ded-6793-47a4-847e-066f3435afa2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:43.795112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634307d-3f6c-49d8-b60f-50384dccb744 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:43.820168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adec39-f2d8-43f0-bc09-2b1b007a021d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:43.822667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adec39-f2d8-43f0-bc09-2b1b007a021d service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:43.824586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adec39-f2d8-43f0-bc09-2b1b007a021d service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:43.826758 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adec39-f2d8-43f0-bc09-2b1b007a021d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:43.831363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d31aaa-0ab9-4bd6-94f9-509a5f2138d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:43.832255 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d31aaa-0ab9-4bd6-94f9-509a5f2138d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:43.832739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d31aaa-0ab9-4bd6-94f9-509a5f2138d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:43.833967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d31aaa-0ab9-4bd6-94f9-509a5f2138d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:43.841458 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adec39-f2d8-43f0-bc09-2b1b007a021d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:43.845447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adec39-f2d8-43f0-bc09-2b1b007a021d service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:43.874384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 894/1768] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:43.874384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90ff8c-7901-4d7e-a2f1-29941f3b87c2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:43.874384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90ff8c-7901-4d7e-a2f1-29941f3b87c2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:43.874384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90ff8c-7901-4d7e-a2f1-29941f3b87c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:43.941974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d31aaa-0ab9-4bd6-94f9-509a5f2138d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:43.942348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-fe90ff8c-7901-4d7e-a2f1-29941f3b87c2 None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:43.943870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fe90ff8c-7901-4d7e-a2f1-29941f3b87c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:43.946089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d31aaa-0ab9-4bd6-94f9-509a5f2138d7 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:43.959555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 875/1769] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:43.960440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d31aaa-0ab9-4bd6-94f9-509a5f2138d7 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:43.973474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d31aaa-0ab9-4bd6-94f9-509a5f2138d7 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:43.976636 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 895/1770] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Sep 16 08:05:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:43.977863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90ff8c-7901-4d7e-a2f1-29941f3b87c2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:43.979954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5062467-6be0-472c-aab3-ee06dd5da980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:43.981135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5062467-6be0-472c-aab3-ee06dd5da980 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:43.981428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5062467-6be0-472c-aab3-ee06dd5da980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:43.981749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5062467-6be0-472c-aab3-ee06dd5da980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:43.982406 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5062467-6be0-472c-aab3-ee06dd5da980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:43.985070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5062467-6be0-472c-aab3-ee06dd5da980 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:44.007697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90ff8c-7901-4d7e-a2f1-29941f3b87c2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:44.016405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 876/1771] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:44.017341 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5062467-6be0-472c-aab3-ee06dd5da980 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:44.022092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923e5b4-4d24-494f-9ac6-6285660a359b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:44.022877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923e5b4-4d24-494f-9ac6-6285660a359b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:44.033645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5062467-6be0-472c-aab3-ee06dd5da980 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:44.036180 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923e5b4-4d24-494f-9ac6-6285660a359b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:44.037807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923e5b4-4d24-494f-9ac6-6285660a359b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:44.038772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923e5b4-4d24-494f-9ac6-6285660a359b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:44.040855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923e5b4-4d24-494f-9ac6-6285660a359b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:44.055722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013aebb5-139a-411e-ba54-e7bf3f3ccac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:44.056472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013aebb5-139a-411e-ba54-e7bf3f3ccac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:44.062206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013aebb5-139a-411e-ba54-e7bf3f3ccac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:44.062206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013aebb5-139a-411e-ba54-e7bf3f3ccac8 admin admin] PATH_INFO: `/v3/projects/0e9a6bbc2cb2480986bf06cfe54476c4/users/17a62ec718b74813a71f5e74f3f4a937/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:44.062206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013aebb5-139a-411e-ba54-e7bf3f3ccac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9a6bbc2cb2480986bf06cfe54476c4, user_id=17a62ec718b74813a71f5e74f3f4a937, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:44.067254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013aebb5-139a-411e-ba54-e7bf3f3ccac8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:44.082313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 896/1772] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:44.086726 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923e5b4-4d24-494f-9ac6-6285660a359b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:44.103241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 877/1773] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:44 2022] PUT /identity/v3/projects/0e9a6bbc2cb2480986bf06cfe54476c4/users/17a62ec718b74813a71f5e74f3f4a937/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:44.116461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923e5b4-4d24-494f-9ac6-6285660a359b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:44.119361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925386e9-8345-471b-906a-93518b5238da None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:44.120794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925386e9-8345-471b-906a-93518b5238da None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:44.120794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925386e9-8345-471b-906a-93518b5238da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:44.231831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-925386e9-8345-471b-906a-93518b5238da None None] MFA Rules not processed for user `17a62ec718b74813a71f5e74f3f4a937`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:44.238103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-925386e9-8345-471b-906a-93518b5238da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:44.320171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022877e-0d18-4820-ac84-cd1c45cb93d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:44.323618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022877e-0d18-4820-ac84-cd1c45cb93d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:44.323912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022877e-0d18-4820-ac84-cd1c45cb93d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:44.324183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022877e-0d18-4820-ac84-cd1c45cb93d4 admin admin] PATH_INFO: `/v3/projects/e8f0589ac89845c2b026663cdb0ad4f3/users/2a26f1da89984e4cb9e361ea536ed53b/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:44.328644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:44.338907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022877e-0d18-4820-ac84-cd1c45cb93d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f0589ac89845c2b026663cdb0ad4f3, user_id=2a26f1da89984e4cb9e361ea536ed53b, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:44.352137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022877e-0d18-4820-ac84-cd1c45cb93d4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:44.390044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 897/1774] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:44 2022] PUT /identity/v3/projects/e8f0589ac89845c2b026663cdb0ad4f3/users/2a26f1da89984e4cb9e361ea536ed53b/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:44.401859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022877e-0d18-4820-ac84-cd1c45cb93d4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:44.455876 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 878/1775] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:44.458155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a449c-be7c-4b21-9575-8c603a65ec4f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:44.459361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a449c-be7c-4b21-9575-8c603a65ec4f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:44.459645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a449c-be7c-4b21-9575-8c603a65ec4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:44.538135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022877e-0d18-4820-ac84-cd1c45cb93d4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:44.545686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd50942-f2ee-4452-a887-d0d3d42c39b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:44.546370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd50942-f2ee-4452-a887-d0d3d42c39b9 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:44.546862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd50942-f2ee-4452-a887-d0d3d42c39b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:44.547067 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd50942-f2ee-4452-a887-d0d3d42c39b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:44.550989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ab8a449c-be7c-4b21-9575-8c603a65ec4f None None] MFA Rules not processed for user `2a26f1da89984e4cb9e361ea536ed53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:44.553505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ab8a449c-be7c-4b21-9575-8c603a65ec4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:44.579625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd50942-f2ee-4452-a887-d0d3d42c39b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:44.582573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd50942-f2ee-4452-a887-d0d3d42c39b9 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:44.652713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 898/1776] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:44.654119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd50942-f2ee-4452-a887-d0d3d42c39b9 service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:44.657710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4cd50942-f2ee-4452-a887-d0d3d42c39b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:44.767672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 879/1777] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:44.775107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a449c-be7c-4b21-9575-8c603a65ec4f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:44.777112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ab8a449c-be7c-4b21-9575-8c603a65ec4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:44.824554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd50942-f2ee-4452-a887-d0d3d42c39b9 service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:44.834532 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51281b0-c839-45fb-b0c3-52bc31f56a97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:44.835443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51281b0-c839-45fb-b0c3-52bc31f56a97 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:44.835878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51281b0-c839-45fb-b0c3-52bc31f56a97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:44.836304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51281b0-c839-45fb-b0c3-52bc31f56a97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:44.914173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:44.950614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a449c-be7c-4b21-9575-8c603a65ec4f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:44.962584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27489927-3f39-4530-a918-28d4090a49c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:44.964895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27489927-3f39-4530-a918-28d4090a49c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:44.964895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27489927-3f39-4530-a918-28d4090a49c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:44.965273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27489927-3f39-4530-a918-28d4090a49c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:44.984213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51281b0-c839-45fb-b0c3-52bc31f56a97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:44.986209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51281b0-c839-45fb-b0c3-52bc31f56a97 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:44.991279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27489927-3f39-4530-a918-28d4090a49c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:44.992744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27489927-3f39-4530-a918-28d4090a49c0 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:45.004663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 880/1778] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:45.012323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 899/1779] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:45.333627 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27489927-3f39-4530-a918-28d4090a49c0 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:45.343507 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27489927-3f39-4530-a918-28d4090a49c0 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:45.348373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7b6b9-d8e8-4178-9566-710cc8a79ed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:45.350132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7b6b9-d8e8-4178-9566-710cc8a79ed3 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:45.350436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7b6b9-d8e8-4178-9566-710cc8a79ed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:45.350747 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7b6b9-d8e8-4178-9566-710cc8a79ed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:45.352991 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cca7b6b9-d8e8-4178-9566-710cc8a79ed3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:45.408441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:45.491513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7b6b9-d8e8-4178-9566-710cc8a79ed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:45.495761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7b6b9-d8e8-4178-9566-710cc8a79ed3 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:45.519251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 881/1780] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:45.675190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51281b0-c839-45fb-b0c3-52bc31f56a97 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:45.679285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e51281b0-c839-45fb-b0c3-52bc31f56a97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:45.891462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51281b0-c839-45fb-b0c3-52bc31f56a97 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:45.905478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:45.906663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:45.908058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:45.909483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] PATH_INFO: `/v3/projects/633fb3e0018848769357421eeb26ba46` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:45.911170 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=633fb3e0018848769357421eeb26ba46)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:45.918470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:46.112181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633fb3e0018848769357421eeb26ba46', 'request_id': 'req-aa41fef5-fbf6-488c-bffa-888b619f1d38'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:46.137385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633fb3e0018848769357421eeb26ba46', 'request_id': 'req-aa41fef5-fbf6-488c-bffa-888b619f1d38'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:46.144947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:46.157351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633fb3e0018848769357421eeb26ba46', 'request_id': 'req-aa41fef5-fbf6-488c-bffa-888b619f1d38'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:46.268177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] The token cache is being invalidate because project 633fb3e0018848769357421eeb26ba46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:46.268586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:46.273354 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:46.279337 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 900/1781] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:45 2022] DELETE /identity/v3/projects/633fb3e0018848769357421eeb26ba46 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:46.284134 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7b6b9-d8e8-4178-9566-710cc8a79ed3 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:46.309693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7b6b9-d8e8-4178-9566-710cc8a79ed3 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:46.419648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:46.421285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:46.421841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:46.422270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] PATH_INFO: `/v3/users/026821febc8c42d8b02c08c03678e1b6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:46.424215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] RBAC: Authorizing `identity:delete_user(user_id=026821febc8c42d8b02c08c03678e1b6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:46.429124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:46.690032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '026821febc8c42d8b02c08c03678e1b6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:46.765090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '026821febc8c42d8b02c08c03678e1b6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:46.777623 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '026821febc8c42d8b02c08c03678e1b6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:46.891117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '026821febc8c42d8b02c08c03678e1b6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:46.895772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '026821febc8c42d8b02c08c03678e1b6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:46.907960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 882/1782] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:46 2022] DELETE /identity/v3/users/026821febc8c42d8b02c08c03678e1b6 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:47.018030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:47.223105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:47.613039 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:47.617013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:47.659263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41fef5-fbf6-488c-bffa-888b619f1d38 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:47.805771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c125de-37e7-4269-88f7-4cc38da39245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:47.807131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c125de-37e7-4269-88f7-4cc38da39245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:47.807575 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c125de-37e7-4269-88f7-4cc38da39245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:47.808311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c125de-37e7-4269-88f7-4cc38da39245 admin admin] PATH_INFO: `/v3/users/c2e6434ca3ce4bcaa38adae095d9c1e4` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:47.810033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c125de-37e7-4269-88f7-4cc38da39245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e6434ca3ce4bcaa38adae095d9c1e4)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:47.812526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c125de-37e7-4269-88f7-4cc38da39245 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:47.906168 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:47.912390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:47.963603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c92d7-6b73-4ca9-bee8-ffde4e65c10d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:47.976905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:47.982276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:47.983040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:47.995877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] PATH_INFO: `/v3/projects/ef38ee76551d4e8a86917c86c29c8c7e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:47.995877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef38ee76551d4e8a86917c86c29c8c7e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:47.995877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:48.119498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:48.156325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b7c125de-37e7-4269-88f7-4cc38da39245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e6434ca3ce4bcaa38adae095d9c1e4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:48.172710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b7c125de-37e7-4269-88f7-4cc38da39245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e6434ca3ce4bcaa38adae095d9c1e4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:48.181900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b7c125de-37e7-4269-88f7-4cc38da39245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e6434ca3ce4bcaa38adae095d9c1e4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:48.304503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b7c125de-37e7-4269-88f7-4cc38da39245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e6434ca3ce4bcaa38adae095d9c1e4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:48.305162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b7c125de-37e7-4269-88f7-4cc38da39245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e6434ca3ce4bcaa38adae095d9c1e4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:48.305400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef38ee76551d4e8a86917c86c29c8c7e', 'request_id': 'req-04a10787-922b-4968-a87d-c1b067e0561e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:48.312304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 901/1783] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:47 2022] DELETE /identity/v3/users/c2e6434ca3ce4bcaa38adae095d9c1e4 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:48.316912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c554bc-45cd-44d5-9729-6528237b2ade None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:48.317422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c554bc-45cd-44d5-9729-6528237b2ade None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:48.317739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c554bc-45cd-44d5-9729-6528237b2ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:48.318514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef38ee76551d4e8a86917c86c29c8c7e', 'request_id': 'req-04a10787-922b-4968-a87d-c1b067e0561e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:48.337738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef38ee76551d4e8a86917c86c29c8c7e', 'request_id': 'req-04a10787-922b-4968-a87d-c1b067e0561e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:48.351094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c8c554bc-45cd-44d5-9729-6528237b2ade None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:48.352721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c8c554bc-45cd-44d5-9729-6528237b2ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:48.398417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:48.443839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] The token cache is being invalidate because project ef38ee76551d4e8a86917c86c29c8c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:48.443839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:48.451106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-04a10787-922b-4968-a87d-c1b067e0561e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:48.453668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 883/1784] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:47 2022] DELETE /identity/v3/projects/ef38ee76551d4e8a86917c86c29c8c7e => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:48.460190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7428e-d183-4526-bd29-7036c14b376b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:48.460712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7428e-d183-4526-bd29-7036c14b376b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:48.461162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7428e-d183-4526-bd29-7036c14b376b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:48.521443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c1f7428e-d183-4526-bd29-7036c14b376b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:48.523246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c1f7428e-d183-4526-bd29-7036c14b376b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:48.534594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 902/1785] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:48.559233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 884/1786] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:48.559342 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c554bc-45cd-44d5-9729-6528237b2ade None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:48.559342 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c8c554bc-45cd-44d5-9729-6528237b2ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:48.564814 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ce802-e592-4c1a-a355-c23a99176789 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:48.565104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ce802-e592-4c1a-a355-c23a99176789 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:48.565421 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ce802-e592-4c1a-a355-c23a99176789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:48.671826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-849ce802-e592-4c1a-a355-c23a99176789 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:48.682167 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-849ce802-e592-4c1a-a355-c23a99176789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:48.724772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 885/1787] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:48.727926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ce802-e592-4c1a-a355-c23a99176789 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:48.761096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ce802-e592-4c1a-a355-c23a99176789 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:48.767417 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ac8bd-6636-41d6-a714-57502a1f4904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:48.767660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ac8bd-6636-41d6-a714-57502a1f4904 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:48.768028 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ac8bd-6636-41d6-a714-57502a1f4904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:48.769752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ac8bd-6636-41d6-a714-57502a1f4904 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:48.769752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ac8bd-6636-41d6-a714-57502a1f4904 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:48.769752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ac8bd-6636-41d6-a714-57502a1f4904 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:48.771760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ac8bd-6636-41d6-a714-57502a1f4904 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:48.788062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c554bc-45cd-44d5-9729-6528237b2ade None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:48.797974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca14f8-d05a-4e80-9bae-a6b52470328e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:48.803454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 886/1788] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:48.803574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca14f8-d05a-4e80-9bae-a6b52470328e service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:48.803574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca14f8-d05a-4e80-9bae-a6b52470328e service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:48.803574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca14f8-d05a-4e80-9bae-a6b52470328e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:48.803772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ac8bd-6636-41d6-a714-57502a1f4904 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:48.821666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca14f8-d05a-4e80-9bae-a6b52470328e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:48.823684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca14f8-d05a-4e80-9bae-a6b52470328e service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:48.850122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 903/1789] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:48.857576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ac8bd-6636-41d6-a714-57502a1f4904 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:48.869732 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0c667-1668-497a-a776-2fd4bdce8aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:48.870758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c667-1668-497a-a776-2fd4bdce8aea admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:48.871064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c667-1668-497a-a776-2fd4bdce8aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:48.871397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c667-1668-497a-a776-2fd4bdce8aea admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:48.873314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0c667-1668-497a-a776-2fd4bdce8aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:48.880086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0c667-1668-497a-a776-2fd4bdce8aea admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:49.029477 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 887/1790] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:48 2022] POST /identity/v3/projects => generated 379 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:49.033822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca14f8-d05a-4e80-9bae-a6b52470328e service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:49.048238 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca14f8-d05a-4e80-9bae-a6b52470328e service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:49.213966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37560810-1868-4cb7-a632-c9250e90daf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:49.214628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:49.215875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37560810-1868-4cb7-a632-c9250e90daf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:49.216172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37560810-1868-4cb7-a632-c9250e90daf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:49.216455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37560810-1868-4cb7-a632-c9250e90daf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:49.217115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37560810-1868-4cb7-a632-c9250e90daf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:49.219131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37560810-1868-4cb7-a632-c9250e90daf0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:49.380262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 904/1791] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:49 2022] POST /identity/v3/users => generated 290 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:49.384736 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0c667-1668-497a-a776-2fd4bdce8aea admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:49.433937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0c667-1668-497a-a776-2fd4bdce8aea admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:49.459728 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8bab3-499b-47a7-b480-15216c4fa7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:49.461381 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8bab3-499b-47a7-b480-15216c4fa7fa admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:49.471189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8bab3-499b-47a7-b480-15216c4fa7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:49.471785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8bab3-499b-47a7-b480-15216c4fa7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:49.486292 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8bab3-499b-47a7-b480-15216c4fa7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:49.493635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8bab3-499b-47a7-b480-15216c4fa7fa admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:49.493635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:49.536394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 888/1792] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:49.541168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37560810-1868-4cb7-a632-c9250e90daf0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:49.562271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37560810-1868-4cb7-a632-c9250e90daf0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:49.576188 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa136375-aa6a-46c8-b7a8-75f4116b6169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:49.577666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa136375-aa6a-46c8-b7a8-75f4116b6169 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:49.578806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa136375-aa6a-46c8-b7a8-75f4116b6169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:49.579253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa136375-aa6a-46c8-b7a8-75f4116b6169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:49.580096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa136375-aa6a-46c8-b7a8-75f4116b6169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:49.587062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa136375-aa6a-46c8-b7a8-75f4116b6169 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:49.633778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 905/1793] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:49.636904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8bab3-499b-47a7-b480-15216c4fa7fa admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:49.659095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8bab3-499b-47a7-b480-15216c4fa7fa admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:49.665908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea94be-d09b-4551-83e8-8ccfe084caa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:49.666612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea94be-d09b-4551-83e8-8ccfe084caa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:49.666945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea94be-d09b-4551-83e8-8ccfe084caa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:49.667237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea94be-d09b-4551-83e8-8ccfe084caa8 admin admin] PATH_INFO: `/v3/projects/0062179f65884eb2a640f0c6a5e3815b/users/1da2a462342a4bb3a04f85609b5f3b59/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:49.694539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea94be-d09b-4551-83e8-8ccfe084caa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0062179f65884eb2a640f0c6a5e3815b, user_id=1da2a462342a4bb3a04f85609b5f3b59, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:49.697969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea94be-d09b-4551-83e8-8ccfe084caa8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:49.791857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 889/1794] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:49 2022] PUT /identity/v3/projects/0062179f65884eb2a640f0c6a5e3815b/users/1da2a462342a4bb3a04f85609b5f3b59/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:49.804690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9eb54-fda7-4d4d-abf8-440dcf012bd6 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:49.805508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9eb54-fda7-4d4d-abf8-440dcf012bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:49.806191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9eb54-fda7-4d4d-abf8-440dcf012bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:49.863808 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-dce9eb54-fda7-4d4d-abf8-440dcf012bd6 None None] MFA Rules not processed for user `1da2a462342a4bb3a04f85609b5f3b59`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:49.865818 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dce9eb54-fda7-4d4d-abf8-440dcf012bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:50.008850 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:50.060129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 906/1795] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:50.064964 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea94be-d09b-4551-83e8-8ccfe084caa8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:50.203569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:50.206201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea94be-d09b-4551-83e8-8ccfe084caa8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:50.267022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2b97f-7c72-4b8a-af05-a58afebf819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:50.267700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2b97f-7c72-4b8a-af05-a58afebf819c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:50.267968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2b97f-7c72-4b8a-af05-a58afebf819c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:50.268245 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2b97f-7c72-4b8a-af05-a58afebf819c admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:50.282917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2b97f-7c72-4b8a-af05-a58afebf819c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:50.282917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2b97f-7c72-4b8a-af05-a58afebf819c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:50.318169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9eb54-fda7-4d4d-abf8-440dcf012bd6 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:50.318169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dce9eb54-fda7-4d4d-abf8-440dcf012bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:50.334430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9eb54-fda7-4d4d-abf8-440dcf012bd6 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:50.340893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:50.352064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:50.352064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:50.352064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] PATH_INFO: `/v3/users/9171d03e8ba94f139472d1a9c48be9d3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:50.352064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9171d03e8ba94f139472d1a9c48be9d3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:50.352064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:50.388210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 890/1796] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:50 2022] POST /identity/v3/projects => generated 379 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:50.392293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2b97f-7c72-4b8a-af05-a58afebf819c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:50.504143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2b97f-7c72-4b8a-af05-a58afebf819c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:50.515427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:50.523125 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de736ea-9dd0-43a1-9eb4-6bccbfffe04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:50.524548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de736ea-9dd0-43a1-9eb4-6bccbfffe04c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:50.527465 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de736ea-9dd0-43a1-9eb4-6bccbfffe04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:50.528098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de736ea-9dd0-43a1-9eb4-6bccbfffe04c admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:50.529038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de736ea-9dd0-43a1-9eb4-6bccbfffe04c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:50.532088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de736ea-9dd0-43a1-9eb4-6bccbfffe04c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:50.607732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9171d03e8ba94f139472d1a9c48be9d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:50.635525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9171d03e8ba94f139472d1a9c48be9d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:50.646403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 891/1797] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:50 2022] POST /identity/v3/users => generated 296 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:05:50.648213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de736ea-9dd0-43a1-9eb4-6bccbfffe04c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:50.654987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9171d03e8ba94f139472d1a9c48be9d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:50.655361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5de736ea-9dd0-43a1-9eb4-6bccbfffe04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:50.673665 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de736ea-9dd0-43a1-9eb4-6bccbfffe04c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:50.679865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:50.682803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:50.682803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:50.682803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] PATH_INFO: `/v3/projects/00acb1faba2d4d28b8e72ea2c435a613` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:50.682803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00acb1faba2d4d28b8e72ea2c435a613)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:50.685670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:50.784272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9171d03e8ba94f139472d1a9c48be9d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:50.784916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9171d03e8ba94f139472d1a9c48be9d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:50.796202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 907/1798] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:50 2022] DELETE /identity/v3/users/9171d03e8ba94f139472d1a9c48be9d3 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:50.797292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:50.799050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:50.840358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00acb1faba2d4d28b8e72ea2c435a613', 'request_id': 'req-6a58324c-babb-467a-80df-5016ce7e2c49'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:50.847982 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00acb1faba2d4d28b8e72ea2c435a613', 'request_id': 'req-6a58324c-babb-467a-80df-5016ce7e2c49'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:50.855893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00acb1faba2d4d28b8e72ea2c435a613', 'request_id': 'req-6a58324c-babb-467a-80df-5016ce7e2c49'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:50.912024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19324817-d8b0-4142-a96c-9ed6b559327b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:50.936989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] The token cache is being invalidate because project 00acb1faba2d4d28b8e72ea2c435a613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:50.937274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:50.937938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6a58324c-babb-467a-80df-5016ce7e2c49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:50.939473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 892/1799] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:50 2022] DELETE /identity/v3/projects/00acb1faba2d4d28b8e72ea2c435a613 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:50.940658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0948f-9361-4aa9-a761-fafb621388e5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:50.940936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0948f-9361-4aa9-a761-fafb621388e5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:50.941212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0948f-9361-4aa9-a761-fafb621388e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:50.965037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c4b0948f-9361-4aa9-a761-fafb621388e5 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:50.966575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c4b0948f-9361-4aa9-a761-fafb621388e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:50.974485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c091e-a04b-4863-bd5e-65094491cdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:50.975147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c091e-a04b-4863-bd5e-65094491cdfe admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:50.975425 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c091e-a04b-4863-bd5e-65094491cdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:50.975695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c091e-a04b-4863-bd5e-65094491cdfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:50.976340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c091e-a04b-4863-bd5e-65094491cdfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:50.978272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c091e-a04b-4863-bd5e-65094491cdfe admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:50.988651 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 908/1800] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:50.989361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c091e-a04b-4863-bd5e-65094491cdfe admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:50.991021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fd2c091e-a04b-4863-bd5e-65094491cdfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:51.010095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c091e-a04b-4863-bd5e-65094491cdfe admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:51.015162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7bb89-0d17-4cea-bc51-456c1dcb5936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:51.015865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7bb89-0d17-4cea-bc51-456c1dcb5936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:51.016251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7bb89-0d17-4cea-bc51-456c1dcb5936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:51.016962 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7bb89-0d17-4cea-bc51-456c1dcb5936 admin admin] PATH_INFO: `/v3/users/2a26f1da89984e4cb9e361ea536ed53b` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:51.017796 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7bb89-0d17-4cea-bc51-456c1dcb5936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a26f1da89984e4cb9e361ea536ed53b)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:51.019896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7bb89-0d17-4cea-bc51-456c1dcb5936 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:51.040374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 893/1801] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:51.041208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0948f-9361-4aa9-a761-fafb621388e5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:51.042494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c4b0948f-9361-4aa9-a761-fafb621388e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:51.054449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0948f-9361-4aa9-a761-fafb621388e5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:51.059233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8618f-0197-42dc-b4fd-ffabaf1f54d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:51.059891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8618f-0197-42dc-b4fd-ffabaf1f54d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:51.060168 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8618f-0197-42dc-b4fd-ffabaf1f54d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:51.060686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8618f-0197-42dc-b4fd-ffabaf1f54d6 admin admin] PATH_INFO: `/v3/projects/8ec1654d167244e89717093d25fbc78c/users/5de47d1e999d4b4a927d9163649913a5/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:51.075021 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8618f-0197-42dc-b4fd-ffabaf1f54d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec1654d167244e89717093d25fbc78c, user_id=5de47d1e999d4b4a927d9163649913a5, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:51.077673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8618f-0197-42dc-b4fd-ffabaf1f54d6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:51.156989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 894/1802] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:51 2022] PUT /identity/v3/projects/8ec1654d167244e89717093d25fbc78c/users/5de47d1e999d4b4a927d9163649913a5/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:51.157854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8618f-0197-42dc-b4fd-ffabaf1f54d6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:51.159404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-99f8618f-0197-42dc-b4fd-ffabaf1f54d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:51.214144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8618f-0197-42dc-b4fd-ffabaf1f54d6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:51.217730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406da0cc-3c76-4cb0-9d00-796121a413c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:51.218387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da0cc-3c76-4cb0-9d00-796121a413c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:51.218667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da0cc-3c76-4cb0-9d00-796121a413c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:51.218966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da0cc-3c76-4cb0-9d00-796121a413c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:51.236962 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dea7bb89-0d17-4cea-bc51-456c1dcb5936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a26f1da89984e4cb9e361ea536ed53b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:51.250700 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dea7bb89-0d17-4cea-bc51-456c1dcb5936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a26f1da89984e4cb9e361ea536ed53b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:51.258309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dea7bb89-0d17-4cea-bc51-456c1dcb5936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a26f1da89984e4cb9e361ea536ed53b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:51.289769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406da0cc-3c76-4cb0-9d00-796121a413c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:51.291244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406da0cc-3c76-4cb0-9d00-796121a413c2 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:51.301601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 895/1803] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:51.303669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406da0cc-3c76-4cb0-9d00-796121a413c2 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:51.314462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406da0cc-3c76-4cb0-9d00-796121a413c2 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:51.320937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3bdf8-bc41-4f7d-8a22-01021f4fe5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:51.321758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3bdf8-bc41-4f7d-8a22-01021f4fe5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:51.322095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3bdf8-bc41-4f7d-8a22-01021f4fe5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:51.322370 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3bdf8-bc41-4f7d-8a22-01021f4fe5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:51.323041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3bdf8-bc41-4f7d-8a22-01021f4fe5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:51.326738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3bdf8-bc41-4f7d-8a22-01021f4fe5c8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:51.338306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 896/1804] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:51.343392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3bdf8-bc41-4f7d-8a22-01021f4fe5c8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:51.345392 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dea7bb89-0d17-4cea-bc51-456c1dcb5936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a26f1da89984e4cb9e361ea536ed53b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:51.346265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dea7bb89-0d17-4cea-bc51-456c1dcb5936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a26f1da89984e4cb9e361ea536ed53b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:51.357802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 909/1805] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:50 2022] DELETE /identity/v3/users/2a26f1da89984e4cb9e361ea536ed53b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:51.358584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3bdf8-bc41-4f7d-8a22-01021f4fe5c8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:51.437577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b6450-ae3b-47c4-9de5-aad36807e910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:51.438529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b6450-ae3b-47c4-9de5-aad36807e910 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:51.438917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b6450-ae3b-47c4-9de5-aad36807e910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:51.439828 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b6450-ae3b-47c4-9de5-aad36807e910 admin admin] PATH_INFO: `/v3/projects/8ec1654d167244e89717093d25fbc78c/users/5de47d1e999d4b4a927d9163649913a5/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:51.442080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b6450-ae3b-47c4-9de5-aad36807e910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec1654d167244e89717093d25fbc78c, user_id=5de47d1e999d4b4a927d9163649913a5, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:51.445968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b6450-ae3b-47c4-9de5-aad36807e910 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:51.459759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 897/1806] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:51 2022] PUT /identity/v3/projects/8ec1654d167244e89717093d25fbc78c/users/5de47d1e999d4b4a927d9163649913a5/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:05:51.470458 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4c9d6-44ea-4180-ab9a-5e33ef09df2a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:51.470959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4c9d6-44ea-4180-ab9a-5e33ef09df2a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:51.471329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4c9d6-44ea-4180-ab9a-5e33ef09df2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:51.533033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-96c4c9d6-44ea-4180-ab9a-5e33ef09df2a None None] MFA Rules not processed for user `5de47d1e999d4b4a927d9163649913a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:51.533033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-96c4c9d6-44ea-4180-ab9a-5e33ef09df2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:51.607008 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 910/1807] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:51.618631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b6450-ae3b-47c4-9de5-aad36807e910 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:51.620061 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9d0b6450-ae3b-47c4-9de5-aad36807e910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:51.684886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b6450-ae3b-47c4-9de5-aad36807e910 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:51.688716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c6540-c3e3-4688-9807-2cf34199587f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:51.689407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6540-c3e3-4688-9807-2cf34199587f service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:51.689693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6540-c3e3-4688-9807-2cf34199587f service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:51.689968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6540-c3e3-4688-9807-2cf34199587f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:51.699570 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c6540-c3e3-4688-9807-2cf34199587f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:51.701089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c6540-c3e3-4688-9807-2cf34199587f service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:51.714162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 898/1808] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:51.852088 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:51.906817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4c9d6-44ea-4180-ab9a-5e33ef09df2a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:51.908441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-96c4c9d6-44ea-4180-ab9a-5e33ef09df2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:51.980514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4c9d6-44ea-4180-ab9a-5e33ef09df2a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:51.985441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:51.989045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:51.989045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:51.989045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] PATH_INFO: `/v3/projects/eb1e7f8f9f344c3cbc5ab982e1127153` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:51.989045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb1e7f8f9f344c3cbc5ab982e1127153)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:51.990500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:52.020451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c6540-c3e3-4688-9807-2cf34199587f service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:52.022022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-312c6540-c3e3-4688-9807-2cf34199587f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:52.060332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c6540-c3e3-4688-9807-2cf34199587f service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:52.060332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:52.060825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:52.060825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:52.061246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] PATH_INFO: `/v3/projects/e8f0589ac89845c2b026663cdb0ad4f3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:52.062319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8f0589ac89845c2b026663cdb0ad4f3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:52.064513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:52.123588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb1e7f8f9f344c3cbc5ab982e1127153', 'request_id': 'req-4d850428-c47f-4902-9e33-33a95f861d44'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.129884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb1e7f8f9f344c3cbc5ab982e1127153', 'request_id': 'req-4d850428-c47f-4902-9e33-33a95f861d44'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.142136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb1e7f8f9f344c3cbc5ab982e1127153', 'request_id': 'req-4d850428-c47f-4902-9e33-33a95f861d44'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.174222 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8f0589ac89845c2b026663cdb0ad4f3', 'request_id': 'req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.181132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8f0589ac89845c2b026663cdb0ad4f3', 'request_id': 'req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.186703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8f0589ac89845c2b026663cdb0ad4f3', 'request_id': 'req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.253572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] The token cache is being invalidate because project eb1e7f8f9f344c3cbc5ab982e1127153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:52.253866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.254500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.256752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 911/1809] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:51 2022] DELETE /identity/v3/projects/eb1e7f8f9f344c3cbc5ab982e1127153 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:52.260557 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:52.261862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:52.269208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:52.324890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d850428-c47f-4902-9e33-33a95f861d44 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:52.336849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] The token cache is being invalidate because project e8f0589ac89845c2b026663cdb0ad4f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:52.337144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.337783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3c9ff41-af89-4afb-bcf4-42d42acc16cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.339724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 899/1810] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:52 2022] DELETE /identity/v3/projects/e8f0589ac89845c2b026663cdb0ad4f3 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:52.344212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595e589-8597-4902-aa85-e3d31b8a5740 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:52.344494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595e589-8597-4902-aa85-e3d31b8a5740 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:52.345117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595e589-8597-4902-aa85-e3d31b8a5740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:52.364456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7595e589-8597-4902-aa85-e3d31b8a5740 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:52.366174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7595e589-8597-4902-aa85-e3d31b8a5740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:52.382268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:52.382938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:52.383223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:52.383496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] PATH_INFO: `/v3/users/17a62ec718b74813a71f5e74f3f4a937` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:52.384800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] RBAC: Authorizing `identity:delete_user(user_id=17a62ec718b74813a71f5e74f3f4a937)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:52.386725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:52.427809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 900/1811] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:52.435649 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf3879-bbf1-4f05-98df-3c3095f09bf3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:52.435940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf3879-bbf1-4f05-98df-3c3095f09bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:52.436216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf3879-bbf1-4f05-98df-3c3095f09bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:52.456304 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-58bf3879-bbf1-4f05-98df-3c3095f09bf3 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:52.457509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-58bf3879-bbf1-4f05-98df-3c3095f09bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:52.465531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 901/1812] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:52.469749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf3879-bbf1-4f05-98df-3c3095f09bf3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:52.479261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf3879-bbf1-4f05-98df-3c3095f09bf3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:52.483827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a220727-74be-49a8-9d3a-c961496b4a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:52.484399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a220727-74be-49a8-9d3a-c961496b4a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:52.484679 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a220727-74be-49a8-9d3a-c961496b4a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:52.484954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a220727-74be-49a8-9d3a-c961496b4a3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:52.485589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a220727-74be-49a8-9d3a-c961496b4a3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:52.485924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a220727-74be-49a8-9d3a-c961496b4a3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:52.487588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a220727-74be-49a8-9d3a-c961496b4a3a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:52.499578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 902/1813] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:05:52.503712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a220727-74be-49a8-9d3a-c961496b4a3a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:52.513144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a220727-74be-49a8-9d3a-c961496b4a3a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:52.517944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420b2b6-14d1-4983-bac6-7c64afa55fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:52.518540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420b2b6-14d1-4983-bac6-7c64afa55fff admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:52.520416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420b2b6-14d1-4983-bac6-7c64afa55fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:52.520416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420b2b6-14d1-4983-bac6-7c64afa55fff admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:52.520416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420b2b6-14d1-4983-bac6-7c64afa55fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:52.526411 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420b2b6-14d1-4983-bac6-7c64afa55fff admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:52.648908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 903/1814] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:52 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:05:52.649960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420b2b6-14d1-4983-bac6-7c64afa55fff admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:52.662579 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17a62ec718b74813a71f5e74f3f4a937'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.686153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17a62ec718b74813a71f5e74f3f4a937'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.706051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17a62ec718b74813a71f5e74f3f4a937'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.736263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420b2b6-14d1-4983-bac6-7c64afa55fff admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:52.743574 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f85fe-b2ee-407d-a1be-2ee74e925ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:52.744268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f85fe-b2ee-407d-a1be-2ee74e925ada admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:52.744751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f85fe-b2ee-407d-a1be-2ee74e925ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:52.745119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f85fe-b2ee-407d-a1be-2ee74e925ada admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:52.745833 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f85fe-b2ee-407d-a1be-2ee74e925ada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:52.747900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f85fe-b2ee-407d-a1be-2ee74e925ada admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:52.808038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17a62ec718b74813a71f5e74f3f4a937'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.808667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17a62ec718b74813a71f5e74f3f4a937'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:52.814980 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 912/1815] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:52 2022] DELETE /identity/v3/users/17a62ec718b74813a71f5e74f3f4a937 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:05:52.840552 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 904/1816] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:52 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:05:52.841762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:52.843428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:52.920966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f85fe-b2ee-407d-a1be-2ee74e925ada admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:52.923026 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ab7f85fe-b2ee-407d-a1be-2ee74e925ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:52.930249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571a9aa-7851-4ba1-b7b3-ab143ce1befe admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:52.936198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c5a68-13a2-4532-8480-ab93c8d9b11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:52.936979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c5a68-13a2-4532-8480-ab93c8d9b11c admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:52.938077 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c5a68-13a2-4532-8480-ab93c8d9b11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:52.938478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c5a68-13a2-4532-8480-ab93c8d9b11c admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:52.943484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c5a68-13a2-4532-8480-ab93c8d9b11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:52.945859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c5a68-13a2-4532-8480-ab93c8d9b11c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:52.981090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 913/1817] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:05:52.984817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c5a68-13a2-4532-8480-ab93c8d9b11c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:53.006266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c5a68-13a2-4532-8480-ab93c8d9b11c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:53.025143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e726dcb-12c1-441d-bb9b-3aa6083ea2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:53.026352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e726dcb-12c1-441d-bb9b-3aa6083ea2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.026829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e726dcb-12c1-441d-bb9b-3aa6083ea2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.027105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e726dcb-12c1-441d-bb9b-3aa6083ea2e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.028186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e726dcb-12c1-441d-bb9b-3aa6083ea2e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:53.030309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e726dcb-12c1-441d-bb9b-3aa6083ea2e1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:53.046556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f85fe-b2ee-407d-a1be-2ee74e925ada admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:53.048866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 914/1818] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:05:53.054883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e726dcb-12c1-441d-bb9b-3aa6083ea2e1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:53.109115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e726dcb-12c1-441d-bb9b-3aa6083ea2e1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:53.117012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac8f5b-c2d8-41fd-a262-d89d6603c3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:53.118431 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac8f5b-c2d8-41fd-a262-d89d6603c3cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.119565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac8f5b-c2d8-41fd-a262-d89d6603c3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.125203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac8f5b-c2d8-41fd-a262-d89d6603c3cb admin admin] PATH_INFO: `/v3/projects/c6688e5e922c4076817f99a601aac258/users/9394a5ded2f04c6b938303a01ec7f62a/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.155619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88aa067-82fd-4fba-a5b7-9d545eec551b tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de47d1e999d4b4a927d9163649913a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec1654d167244e89717093d25fbc78c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:53.159434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88aa067-82fd-4fba-a5b7-9d545eec551b tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.159631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88aa067-82fd-4fba-a5b7-9d545eec551b tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.159929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88aa067-82fd-4fba-a5b7-9d545eec551b tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.160889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88aa067-82fd-4fba-a5b7-9d545eec551b tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:53.161198 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88aa067-82fd-4fba-a5b7-9d545eec551b tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:53.162903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88aa067-82fd-4fba-a5b7-9d545eec551b tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:53.165896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac8f5b-c2d8-41fd-a262-d89d6603c3cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6688e5e922c4076817f99a601aac258, user_id=9394a5ded2f04c6b938303a01ec7f62a, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:53.168818 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac8f5b-c2d8-41fd-a262-d89d6603c3cb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:53.182726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 905/1819] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:05:53.187914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88aa067-82fd-4fba-a5b7-9d545eec551b tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:53.202738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88aa067-82fd-4fba-a5b7-9d545eec551b tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:53.213223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8c3a1-84b3-4792-9c2b-2e736e2ce260 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de47d1e999d4b4a927d9163649913a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec1654d167244e89717093d25fbc78c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:53.215554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8c3a1-84b3-4792-9c2b-2e736e2ce260 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.215554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8c3a1-84b3-4792-9c2b-2e736e2ce260 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.215554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8c3a1-84b3-4792-9c2b-2e736e2ce260 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.215554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8c3a1-84b3-4792-9c2b-2e736e2ce260 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:53.217228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8c3a1-84b3-4792-9c2b-2e736e2ce260 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:53.269443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 915/1820] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:53 2022] PUT /identity/v3/projects/c6688e5e922c4076817f99a601aac258/users/9394a5ded2f04c6b938303a01ec7f62a/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:05:53.280168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18bd64-a326-4848-9f00-20dc73000e8c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.280457 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18bd64-a326-4848-9f00-20dc73000e8c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.280730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18bd64-a326-4848-9f00-20dc73000e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.310264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9a18bd64-a326-4848-9f00-20dc73000e8c None None] MFA Rules not processed for user `9394a5ded2f04c6b938303a01ec7f62a`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:53.311515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9a18bd64-a326-4848-9f00-20dc73000e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:53.341609 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 906/1821] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:53 2022] POST /identity/v3/projects => generated 353 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:53.345582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8c3a1-84b3-4792-9c2b-2e736e2ce260 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:53.398082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 916/1822] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:53.411330 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18bd64-a326-4848-9f00-20dc73000e8c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:53.413346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9a18bd64-a326-4848-9f00-20dc73000e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:53.415540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8c3a1-84b3-4792-9c2b-2e736e2ce260 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:53.420596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2dec2-fa7f-4ceb-b0e8-7934a14d94df tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de47d1e999d4b4a927d9163649913a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec1654d167244e89717093d25fbc78c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:53.421244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2dec2-fa7f-4ceb-b0e8-7934a14d94df tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.421519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2dec2-fa7f-4ceb-b0e8-7934a14d94df tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.421798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2dec2-fa7f-4ceb-b0e8-7934a14d94df tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.422457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2dec2-fa7f-4ceb-b0e8-7934a14d94df tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:53.422897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2dec2-fa7f-4ceb-b0e8-7934a14d94df tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:53.424594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2dec2-fa7f-4ceb-b0e8-7934a14d94df tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:53.446039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 907/1823] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:53.473473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2dec2-fa7f-4ceb-b0e8-7934a14d94df tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:53.475274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f1b2dec2-fa7f-4ceb-b0e8-7934a14d94df tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:53.477909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18bd64-a326-4848-9f00-20dc73000e8c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:53.534520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0f60b-09b8-4b4d-9ad3-280cfc89ecf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:53.535164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0f60b-09b8-4b4d-9ad3-280cfc89ecf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.535702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0f60b-09b8-4b4d-9ad3-280cfc89ecf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.535976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0f60b-09b8-4b4d-9ad3-280cfc89ecf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.545802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2dec2-fa7f-4ceb-b0e8-7934a14d94df tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:53.550416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:53.551071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.551348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.551621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] PATH_INFO: `/v3/projects/0e9a6bbc2cb2480986bf06cfe54476c4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.554757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e9a6bbc2cb2480986bf06cfe54476c4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:53.555345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:53.592304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0f60b-09b8-4b4d-9ad3-280cfc89ecf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:53.593738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0f60b-09b8-4b4d-9ad3-280cfc89ecf2 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:53.602936 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 917/1824] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:05:53.603773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0f60b-09b8-4b4d-9ad3-280cfc89ecf2 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:53.622745 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0f60b-09b8-4b4d-9ad3-280cfc89ecf2 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:53.627743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de47d1e999d4b4a927d9163649913a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec1654d167244e89717093d25fbc78c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:53.628398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.628680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.639918 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] PATH_INFO: `/v3/projects/43e60ce5d5c94d729b901e5e74e3d3b4` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.639918 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: Authorizing `identity:delete_project(project_id=43e60ce5d5c94d729b901e5e74e3d3b4)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:53.639918 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:53.677611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e9a6bbc2cb2480986bf06cfe54476c4', 'request_id': 'req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:53.683972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e9a6bbc2cb2480986bf06cfe54476c4', 'request_id': 'req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:53.689779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e9a6bbc2cb2480986bf06cfe54476c4', 'request_id': 'req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:53.745752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43e60ce5d5c94d729b901e5e74e3d3b4', 'request_id': 'req-d9c331fe-f618-4c36-a7c2-97a0f43e3410'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:53.753011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43e60ce5d5c94d729b901e5e74e3d3b4', 'request_id': 'req-d9c331fe-f618-4c36-a7c2-97a0f43e3410'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:53.761821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43e60ce5d5c94d729b901e5e74e3d3b4', 'request_id': 'req-d9c331fe-f618-4c36-a7c2-97a0f43e3410'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:53.764985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] The token cache is being invalidate because project 0e9a6bbc2cb2480986bf06cfe54476c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:53.765293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:53.766011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:53.767669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 908/1825] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:53 2022] DELETE /identity/v3/projects/0e9a6bbc2cb2480986bf06cfe54476c4 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:05:53.768451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:53.770304 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:53.840204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c25ab-3a44-4349-97a9-0a4e31c4c7fa admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:53.859625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] The token cache is being invalidate because project 43e60ce5d5c94d729b901e5e74e3d3b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:53.860027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:53.867103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d9c331fe-f618-4c36-a7c2-97a0f43e3410 tempest-VolumeQuotaClassesTest-362025080 tempest-VolumeQuotaClassesTest-362025080-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:53.868746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 918/1826] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:53 2022] DELETE /identity/v3/projects/43e60ce5d5c94d729b901e5e74e3d3b4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:53.870047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09cdba-8901-4e30-a6d0-fbf7a770e333 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.870323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09cdba-8901-4e30-a6d0-fbf7a770e333 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.870597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09cdba-8901-4e30-a6d0-fbf7a770e333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.893041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5e09cdba-8901-4e30-a6d0-fbf7a770e333 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:53.894336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5e09cdba-8901-4e30-a6d0-fbf7a770e333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:53.921676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df357d-72e8-467c-88c9-e70462822f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:53.922522 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df357d-72e8-467c-88c9-e70462822f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.923303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df357d-72e8-467c-88c9-e70462822f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.924099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df357d-72e8-467c-88c9-e70462822f20 admin admin] PATH_INFO: `/v3/users/9394a5ded2f04c6b938303a01ec7f62a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.925388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df357d-72e8-467c-88c9-e70462822f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9394a5ded2f04c6b938303a01ec7f62a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:53.928445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df357d-72e8-467c-88c9-e70462822f20 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:53.933390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:53.965338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 919/1827] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:53.972828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e260873-f6b2-4850-9b0e-9cb2e295771c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:53.973117 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e260873-f6b2-4850-9b0e-9cb2e295771c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:53.973389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e260873-f6b2-4850-9b0e-9cb2e295771c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:53.994961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9e260873-f6b2-4850-9b0e-9cb2e295771c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:53.996173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9e260873-f6b2-4850-9b0e-9cb2e295771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:54.004393 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 920/1828] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:54.009031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e260873-f6b2-4850-9b0e-9cb2e295771c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:54.018044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e260873-f6b2-4850-9b0e-9cb2e295771c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:54.023255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c5095-04ac-4aef-8536-bfcc5505a3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:54.023906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c5095-04ac-4aef-8536-bfcc5505a3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.024132 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c5095-04ac-4aef-8536-bfcc5505a3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.024414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c5095-04ac-4aef-8536-bfcc5505a3f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.025057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c5095-04ac-4aef-8536-bfcc5505a3f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:54.025364 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c5095-04ac-4aef-8536-bfcc5505a3f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:54.027017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c5095-04ac-4aef-8536-bfcc5505a3f7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:54.038967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 921/1829] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:05:54.043129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c5095-04ac-4aef-8536-bfcc5505a3f7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:54.052544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c5095-04ac-4aef-8536-bfcc5505a3f7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:54.057211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa049f-e95a-4d16-ba95-a648b824fe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:54.057794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa049f-e95a-4d16-ba95-a648b824fe76 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.058072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa049f-e95a-4d16-ba95-a648b824fe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.058347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa049f-e95a-4d16-ba95-a648b824fe76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.059029 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa049f-e95a-4d16-ba95-a648b824fe76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:54.060935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa049f-e95a-4d16-ba95-a648b824fe76 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:54.155858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-37df357d-72e8-467c-88c9-e70462822f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9394a5ded2f04c6b938303a01ec7f62a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:54.169464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-37df357d-72e8-467c-88c9-e70462822f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9394a5ded2f04c6b938303a01ec7f62a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:54.174063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 922/1830] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:54 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:05:54.178114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa049f-e95a-4d16-ba95-a648b824fe76 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:54.180698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-37df357d-72e8-467c-88c9-e70462822f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9394a5ded2f04c6b938303a01ec7f62a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:54.220060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:54.245475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa049f-e95a-4d16-ba95-a648b824fe76 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:54.248529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-37df357d-72e8-467c-88c9-e70462822f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9394a5ded2f04c6b938303a01ec7f62a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:54.249249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-37df357d-72e8-467c-88c9-e70462822f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9394a5ded2f04c6b938303a01ec7f62a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:54.250546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50ac41-a504-4d46-bcce-01ab72c58933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:54.251265 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50ac41-a504-4d46-bcce-01ab72c58933 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.251473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50ac41-a504-4d46-bcce-01ab72c58933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.251805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50ac41-a504-4d46-bcce-01ab72c58933 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.253094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50ac41-a504-4d46-bcce-01ab72c58933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:54.255262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50ac41-a504-4d46-bcce-01ab72c58933 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:54.263146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 909/1831] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:53 2022] DELETE /identity/v3/users/9394a5ded2f04c6b938303a01ec7f62a => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:54.264889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b5a91-6b83-4a50-a843-b3b58d32cce6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.265229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b5a91-6b83-4a50-a843-b3b58d32cce6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.265507 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b5a91-6b83-4a50-a843-b3b58d32cce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.285712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0f1b5a91-6b83-4a50-a843-b3b58d32cce6 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:54.286932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0f1b5a91-6b83-4a50-a843-b3b58d32cce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:54.347189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 910/1832] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:54.358799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b5a91-6b83-4a50-a843-b3b58d32cce6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:54.367198 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0f1b5a91-6b83-4a50-a843-b3b58d32cce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:54.376647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 923/1833] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:54 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:05:54.377747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50ac41-a504-4d46-bcce-01ab72c58933 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:54.382051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ab50ac41-a504-4d46-bcce-01ab72c58933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:54.398404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50ac41-a504-4d46-bcce-01ab72c58933 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:54.407112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca31fb-176b-4fdb-98b0-ce16becc921d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:54.408518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca31fb-176b-4fdb-98b0-ce16becc921d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.408860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca31fb-176b-4fdb-98b0-ce16becc921d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.414064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca31fb-176b-4fdb-98b0-ce16becc921d admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.414872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca31fb-176b-4fdb-98b0-ce16becc921d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:54.416737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca31fb-176b-4fdb-98b0-ce16becc921d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:54.428366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 924/1834] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:54.432386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca31fb-176b-4fdb-98b0-ce16becc921d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:54.436575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b5a91-6b83-4a50-a843-b3b58d32cce6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:54.441404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce578e-45ff-4769-942b-a3ac1039b182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:54.442361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce578e-45ff-4769-942b-a3ac1039b182 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.442361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce578e-45ff-4769-942b-a3ac1039b182 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.442573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce578e-45ff-4769-942b-a3ac1039b182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.446536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca31fb-176b-4fdb-98b0-ce16becc921d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:54.451589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edac38-d521-4822-9232-979a266b7738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:54.452338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edac38-d521-4822-9232-979a266b7738 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.452722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edac38-d521-4822-9232-979a266b7738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.452818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce578e-45ff-4769-942b-a3ac1039b182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:54.453171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edac38-d521-4822-9232-979a266b7738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.453921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edac38-d521-4822-9232-979a266b7738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:54.454357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce578e-45ff-4769-942b-a3ac1039b182 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:54.455774 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edac38-d521-4822-9232-979a266b7738 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:54.465525 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 911/1835] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:05:54.470735 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 925/1836] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:05:54.471687 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce578e-45ff-4769-942b-a3ac1039b182 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:54.481937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce578e-45ff-4769-942b-a3ac1039b182 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:54.486872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a319e-ef1c-478c-95b3-89ccc4359c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:54.487531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a319e-ef1c-478c-95b3-89ccc4359c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.487931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a319e-ef1c-478c-95b3-89ccc4359c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.488219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a319e-ef1c-478c-95b3-89ccc4359c0a admin admin] PATH_INFO: `/v3/projects/c634aa313ef04e1eb8b703542887fcd5/users/943da8c557ae4bcc9451e0eb290744f9/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.507734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a319e-ef1c-478c-95b3-89ccc4359c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c634aa313ef04e1eb8b703542887fcd5, user_id=943da8c557ae4bcc9451e0eb290744f9, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:54.511415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a319e-ef1c-478c-95b3-89ccc4359c0a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:54.514860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edac38-d521-4822-9232-979a266b7738 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:54.516688 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-06edac38-d521-4822-9232-979a266b7738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:54.529955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edac38-d521-4822-9232-979a266b7738 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:54.535753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27f80c-ccbb-4b6a-8e1d-d18cc86557ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:54.536487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27f80c-ccbb-4b6a-8e1d-d18cc86557ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.536799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27f80c-ccbb-4b6a-8e1d-d18cc86557ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.537076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27f80c-ccbb-4b6a-8e1d-d18cc86557ce admin admin] PATH_INFO: `/v3/users/1da2a462342a4bb3a04f85609b5f3b59` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.538202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27f80c-ccbb-4b6a-8e1d-d18cc86557ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=1da2a462342a4bb3a04f85609b5f3b59)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:54.540237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27f80c-ccbb-4b6a-8e1d-d18cc86557ce admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:54.605957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 912/1837] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:54 2022] PUT /identity/v3/projects/c634aa313ef04e1eb8b703542887fcd5/users/943da8c557ae4bcc9451e0eb290744f9/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:05:54.622742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d31531-2502-4e2c-97af-2389703902f4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.623037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d31531-2502-4e2c-97af-2389703902f4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.623315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d31531-2502-4e2c-97af-2389703902f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.650352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-54d31531-2502-4e2c-97af-2389703902f4 None None] MFA Rules not processed for user `943da8c557ae4bcc9451e0eb290744f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:54.651582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-54d31531-2502-4e2c-97af-2389703902f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:54.717289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 913/1838] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:54.721936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d31531-2502-4e2c-97af-2389703902f4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:54.789460 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ef27f80c-ccbb-4b6a-8e1d-d18cc86557ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1da2a462342a4bb3a04f85609b5f3b59'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:54.795914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d31531-2502-4e2c-97af-2389703902f4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:54.801816 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a506e74-182c-414d-b6fe-9247c009a7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:54.802270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ef27f80c-ccbb-4b6a-8e1d-d18cc86557ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1da2a462342a4bb3a04f85609b5f3b59'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:54.803499 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a506e74-182c-414d-b6fe-9247c009a7b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.803944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a506e74-182c-414d-b6fe-9247c009a7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.804413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a506e74-182c-414d-b6fe-9247c009a7b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.805188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a506e74-182c-414d-b6fe-9247c009a7b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:54.807510 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a506e74-182c-414d-b6fe-9247c009a7b9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:54.811128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ef27f80c-ccbb-4b6a-8e1d-d18cc86557ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1da2a462342a4bb3a04f85609b5f3b59'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:54.887881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ef27f80c-ccbb-4b6a-8e1d-d18cc86557ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1da2a462342a4bb3a04f85609b5f3b59'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:54.888730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ef27f80c-ccbb-4b6a-8e1d-d18cc86557ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1da2a462342a4bb3a04f85609b5f3b59'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:54.899327 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 926/1839] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:54 2022] DELETE /identity/v3/users/1da2a462342a4bb3a04f85609b5f3b59 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:05:54.899949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383893db-7901-4404-905c-01f5b4f5a53d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:54.900244 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383893db-7901-4404-905c-01f5b4f5a53d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:54.900526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383893db-7901-4404-905c-01f5b4f5a53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:54.919976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-383893db-7901-4404-905c-01f5b4f5a53d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:54.921216 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-383893db-7901-4404-905c-01f5b4f5a53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:54.929656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 914/1840] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:54 2022] POST /identity/v3/projects => generated 367 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:05:54.930787 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a506e74-182c-414d-b6fe-9247c009a7b9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:54.932590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4a506e74-182c-414d-b6fe-9247c009a7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:55.004520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 927/1841] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:55.005537 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383893db-7901-4404-905c-01f5b4f5a53d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.006966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-383893db-7901-4404-905c-01f5b4f5a53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:55.015011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a506e74-182c-414d-b6fe-9247c009a7b9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.022033 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.022882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.023449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.023735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] PATH_INFO: `/v3/projects/c6688e5e922c4076817f99a601aac258` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.025843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6688e5e922c4076817f99a601aac258)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.030596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.030904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383893db-7901-4404-905c-01f5b4f5a53d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.036054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412dcce-eeea-40cb-ac49-04ca035b481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.037725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412dcce-eeea-40cb-ac49-04ca035b481f admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.037725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412dcce-eeea-40cb-ac49-04ca035b481f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.038274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412dcce-eeea-40cb-ac49-04ca035b481f admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.039003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412dcce-eeea-40cb-ac49-04ca035b481f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.043807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412dcce-eeea-40cb-ac49-04ca035b481f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.129879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 928/1842] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:55 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:55.130813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412dcce-eeea-40cb-ac49-04ca035b481f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.134006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3412dcce-eeea-40cb-ac49-04ca035b481f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:55.154058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6688e5e922c4076817f99a601aac258', 'request_id': 'req-f5d0ef29-84d1-455b-9031-020313757257'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:55.161213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6688e5e922c4076817f99a601aac258', 'request_id': 'req-f5d0ef29-84d1-455b-9031-020313757257'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:55.171478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6688e5e922c4076817f99a601aac258', 'request_id': 'req-f5d0ef29-84d1-455b-9031-020313757257'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:55.205954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412dcce-eeea-40cb-ac49-04ca035b481f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.250875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] The token cache is being invalidate because project c6688e5e922c4076817f99a601aac258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:55.251374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:55.251847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:55.253448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 915/1843] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:54 2022] DELETE /identity/v3/projects/c6688e5e922c4076817f99a601aac258 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:55.254222 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.254381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bad208-f096-4fa3-8275-869090e494aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.255042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bad208-f096-4fa3-8275-869090e494aa service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.255353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bad208-f096-4fa3-8275-869090e494aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.255562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:55.255649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bad208-f096-4fa3-8275-869090e494aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.258292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-98bad208-f096-4fa3-8275-869090e494aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:55.320833 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0ef29-84d1-455b-9031-020313757257 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.323754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bad208-f096-4fa3-8275-869090e494aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.325316 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bad208-f096-4fa3-8275-869090e494aa service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.325495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee1384-e055-4d47-a61b-6ad2b09c4334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.326104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee1384-e055-4d47-a61b-6ad2b09c4334 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.326383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee1384-e055-4d47-a61b-6ad2b09c4334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.326652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee1384-e055-4d47-a61b-6ad2b09c4334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.327322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee1384-e055-4d47-a61b-6ad2b09c4334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.328977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee1384-e055-4d47-a61b-6ad2b09c4334 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.337190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 929/1844] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:55.340801 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 916/1845] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:55.345468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34e0f0-0922-457e-bfe0-4d16c831823d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.345771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34e0f0-0922-457e-bfe0-4d16c831823d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.346055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34e0f0-0922-457e-bfe0-4d16c831823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.347722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee1384-e055-4d47-a61b-6ad2b09c4334 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.356225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:55.362027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee1384-e055-4d47-a61b-6ad2b09c4334 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.367687 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1a34e0f0-0922-457e-bfe0-4d16c831823d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:55.367840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8186d-5093-4764-9778-765be49a82c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.368911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1a34e0f0-0922-457e-bfe0-4d16c831823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:55.369142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8186d-5093-4764-9778-765be49a82c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.369489 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8186d-5093-4764-9778-765be49a82c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.369862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8186d-5093-4764-9778-765be49a82c7 admin admin] PATH_INFO: `/v3/projects/931f90a116d740ef9288b4333a7bdcd4/users/711140ebf89242dba466dd868b74707a/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.384196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 930/1846] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:55.387994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4f786-b0e2-44c9-a68b-cff24c8fdeed None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.388737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4f786-b0e2-44c9-a68b-cff24c8fdeed None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.388737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4f786-b0e2-44c9-a68b-cff24c8fdeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.390705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f8186d-5093-4764-9778-765be49a82c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=931f90a116d740ef9288b4333a7bdcd4, user_id=711140ebf89242dba466dd868b74707a, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.395491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f8186d-5093-4764-9778-765be49a82c7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.413117 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e9c4f786-b0e2-44c9-a68b-cff24c8fdeed None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:55.414517 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e9c4f786-b0e2-44c9-a68b-cff24c8fdeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:55.422899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 931/1847] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:55.427774 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4f786-b0e2-44c9-a68b-cff24c8fdeed None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.438764 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4f786-b0e2-44c9-a68b-cff24c8fdeed None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.444736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cc358-451a-41df-9b0e-f49b3b7fd6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.445400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cc358-451a-41df-9b0e-f49b3b7fd6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.445770 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cc358-451a-41df-9b0e-f49b3b7fd6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.446276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cc358-451a-41df-9b0e-f49b3b7fd6a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.446950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cc358-451a-41df-9b0e-f49b3b7fd6a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:55.447289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cc358-451a-41df-9b0e-f49b3b7fd6a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.448905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cc358-451a-41df-9b0e-f49b3b7fd6a2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.462723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 932/1848] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:05:55.465777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cc358-451a-41df-9b0e-f49b3b7fd6a2 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.476727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cc358-451a-41df-9b0e-f49b3b7fd6a2 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.482736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca408cdd-008d-4c17-bd14-db529b630f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.483341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408cdd-008d-4c17-bd14-db529b630f7d admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.483621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408cdd-008d-4c17-bd14-db529b630f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.483895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408cdd-008d-4c17-bd14-db529b630f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.484534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca408cdd-008d-4c17-bd14-db529b630f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.486583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca408cdd-008d-4c17-bd14-db529b630f7d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.534037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 917/1849] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:55 2022] PUT /identity/v3/projects/931f90a116d740ef9288b4333a7bdcd4/users/711140ebf89242dba466dd868b74707a/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:55.538066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8186d-5093-4764-9778-765be49a82c7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.576490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 933/1850] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:55 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:55.580750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca408cdd-008d-4c17-bd14-db529b630f7d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.613345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8186d-5093-4764-9778-765be49a82c7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.625523 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:55.675569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca408cdd-008d-4c17-bd14-db529b630f7d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.681366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec2c75-db09-4bdc-be6e-aa452543e1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.682055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec2c75-db09-4bdc-be6e-aa452543e1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.682854 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec2c75-db09-4bdc-be6e-aa452543e1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.683134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec2c75-db09-4bdc-be6e-aa452543e1e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.684439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec2c75-db09-4bdc-be6e-aa452543e1e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.687466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec2c75-db09-4bdc-be6e-aa452543e1e8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.696347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9a35a-c4bb-48b5-98a9-500c1d270154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.697879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9a35a-c4bb-48b5-98a9-500c1d270154 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.698338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9a35a-c4bb-48b5-98a9-500c1d270154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.698961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9a35a-c4bb-48b5-98a9-500c1d270154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.701290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9a35a-c4bb-48b5-98a9-500c1d270154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.704468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9a35a-c4bb-48b5-98a9-500c1d270154 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.715613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 918/1851] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:05:55.719701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9a35a-c4bb-48b5-98a9-500c1d270154 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.733356 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9a35a-c4bb-48b5-98a9-500c1d270154 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.739277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9de65f-1ab2-4d65-9ecb-42a79e7e2b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.740064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9de65f-1ab2-4d65-9ecb-42a79e7e2b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.740539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9de65f-1ab2-4d65-9ecb-42a79e7e2b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.742977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9de65f-1ab2-4d65-9ecb-42a79e7e2b62 admin admin] PATH_INFO: `/v3/projects/931f90a116d740ef9288b4333a7bdcd4/users/711140ebf89242dba466dd868b74707a/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.745373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9de65f-1ab2-4d65-9ecb-42a79e7e2b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=931f90a116d740ef9288b4333a7bdcd4, user_id=711140ebf89242dba466dd868b74707a, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.748210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9de65f-1ab2-4d65-9ecb-42a79e7e2b62 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.759009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 919/1852] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:55 2022] PUT /identity/v3/projects/931f90a116d740ef9288b4333a7bdcd4/users/711140ebf89242dba466dd868b74707a/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:05:55.770590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e314dd-7328-4614-8bea-8d103a783062 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.770899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e314dd-7328-4614-8bea-8d103a783062 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.771698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e314dd-7328-4614-8bea-8d103a783062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.799481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f3e314dd-7328-4614-8bea-8d103a783062 None None] MFA Rules not processed for user `711140ebf89242dba466dd868b74707a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:55.800626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f3e314dd-7328-4614-8bea-8d103a783062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:55.811401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 934/1853] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:55 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:55.812702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec2c75-db09-4bdc-be6e-aa452543e1e8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.822954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec2c75-db09-4bdc-be6e-aa452543e1e8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.877335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 920/1854] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:05:55.878282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e314dd-7328-4614-8bea-8d103a783062 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.880115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f3e314dd-7328-4614-8bea-8d103a783062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:55.890885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eec5c-559c-47dc-b712-154b8d11a620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.891586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eec5c-559c-47dc-b712-154b8d11a620 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.892256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eec5c-559c-47dc-b712-154b8d11a620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.892256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eec5c-559c-47dc-b712-154b8d11a620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.892829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340eec5c-559c-47dc-b712-154b8d11a620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.894431 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340eec5c-559c-47dc-b712-154b8d11a620 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.907846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 935/1855] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:05:55.907846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eec5c-559c-47dc-b712-154b8d11a620 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:55.907846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-340eec5c-559c-47dc-b712-154b8d11a620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:55.948207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e314dd-7328-4614-8bea-8d103a783062 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.953214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.953866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.954144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.954415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] PATH_INFO: `/v3/projects/0062179f65884eb2a640f0c6a5e3815b` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.955580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0062179f65884eb2a640f0c6a5e3815b)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.957537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.966044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eec5c-559c-47dc-b712-154b8d11a620 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:55.969756 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee107e-b443-4510-b96e-a33c8956f3ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:55.970409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee107e-b443-4510-b96e-a33c8956f3ff service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:55.970715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee107e-b443-4510-b96e-a33c8956f3ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:55.970990 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee107e-b443-4510-b96e-a33c8956f3ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:55.980983 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee107e-b443-4510-b96e-a33c8956f3ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:55.982427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee107e-b443-4510-b96e-a33c8956f3ff service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:55.992467 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 936/1856] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:55.994305 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee107e-b443-4510-b96e-a33c8956f3ff service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:56.000470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:56.006353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee107e-b443-4510-b96e-a33c8956f3ff service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:56.011636 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b88c2a-2fe5-475e-8185-55fe9827ca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:56.012369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b88c2a-2fe5-475e-8185-55fe9827ca86 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:56.012748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b88c2a-2fe5-475e-8185-55fe9827ca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:56.013122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b88c2a-2fe5-475e-8185-55fe9827ca86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:56.015303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b88c2a-2fe5-475e-8185-55fe9827ca86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:56.016059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b88c2a-2fe5-475e-8185-55fe9827ca86 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:56.027861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 937/1857] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:56.032667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b88c2a-2fe5-475e-8185-55fe9827ca86 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:56.042415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0062179f65884eb2a640f0c6a5e3815b', 'request_id': 'req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:56.044385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b88c2a-2fe5-475e-8185-55fe9827ca86 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:56.049097 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0062179f65884eb2a640f0c6a5e3815b', 'request_id': 'req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:56.056866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0062179f65884eb2a640f0c6a5e3815b', 'request_id': 'req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:56.102100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca87747-1c97-43f0-8f2c-1e5abadfaaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:56.102905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca87747-1c97-43f0-8f2c-1e5abadfaaef admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:56.103184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca87747-1c97-43f0-8f2c-1e5abadfaaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:56.108536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca87747-1c97-43f0-8f2c-1e5abadfaaef admin admin] PATH_INFO: `/v3/projects/e4834019a5e44a0388220402d877e997/users/2e4fdaf1205c4816b5c477edc2097ddb/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:56.118426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca87747-1c97-43f0-8f2c-1e5abadfaaef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4834019a5e44a0388220402d877e997, user_id=2e4fdaf1205c4816b5c477edc2097ddb, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:56.121140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca87747-1c97-43f0-8f2c-1e5abadfaaef admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:56.128120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:56.162023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] The token cache is being invalidate because project 0062179f65884eb2a640f0c6a5e3815b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:56.162431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:56.163203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:56.171744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 921/1858] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:55 2022] DELETE /identity/v3/projects/0062179f65884eb2a640f0c6a5e3815b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:56.171744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:56.171744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:56.235465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 938/1859] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:56 2022] PUT /identity/v3/projects/e4834019a5e44a0388220402d877e997/users/2e4fdaf1205c4816b5c477edc2097ddb/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:56.235465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca87747-1c97-43f0-8f2c-1e5abadfaaef admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:56.235465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8ca87747-1c97-43f0-8f2c-1e5abadfaaef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:56.246268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6e3aa-5a7a-4da6-b38a-f9fece53cb92 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:56.315381 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e652b16-7c75-4064-88d4-e01ad7288b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:56.316028 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e652b16-7c75-4064-88d4-e01ad7288b5a service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:56.316213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e652b16-7c75-4064-88d4-e01ad7288b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:56.316485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e652b16-7c75-4064-88d4-e01ad7288b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:56.318348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6e652b16-7c75-4064-88d4-e01ad7288b5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:56.320922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca87747-1c97-43f0-8f2c-1e5abadfaaef admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:56.325863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b9a66-1231-4b03-99c4-25d7f25bd262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:56.327010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b9a66-1231-4b03-99c4-25d7f25bd262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:56.327387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b9a66-1231-4b03-99c4-25d7f25bd262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:56.327760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b9a66-1231-4b03-99c4-25d7f25bd262 admin admin] PATH_INFO: `/v3/users/5de47d1e999d4b4a927d9163649913a5` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:56.328999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b9a66-1231-4b03-99c4-25d7f25bd262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de47d1e999d4b4a927d9163649913a5)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:56.331115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b9a66-1231-4b03-99c4-25d7f25bd262 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:56.383138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e652b16-7c75-4064-88d4-e01ad7288b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:56.384823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e652b16-7c75-4064-88d4-e01ad7288b5a service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:56.396038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 922/1860] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:56.397617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74903b-5b12-4419-8bf2-17f930d16d74 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:56.398050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74903b-5b12-4419-8bf2-17f930d16d74 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:56.398483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74903b-5b12-4419-8bf2-17f930d16d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:56.434041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8f74903b-5b12-4419-8bf2-17f930d16d74 None None] MFA Rules not processed for user `2e4fdaf1205c4816b5c477edc2097ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:56.438467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8f74903b-5b12-4419-8bf2-17f930d16d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:56.512131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 923/1861] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:56.524599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74903b-5b12-4419-8bf2-17f930d16d74 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:56.526474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8f74903b-5b12-4419-8bf2-17f930d16d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:56.550206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-eb3b9a66-1231-4b03-99c4-25d7f25bd262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de47d1e999d4b4a927d9163649913a5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:56.574769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-eb3b9a66-1231-4b03-99c4-25d7f25bd262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de47d1e999d4b4a927d9163649913a5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:56.595123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-eb3b9a66-1231-4b03-99c4-25d7f25bd262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de47d1e999d4b4a927d9163649913a5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:56.634075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74903b-5b12-4419-8bf2-17f930d16d74 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:56.637852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc4133-c741-45aa-b44a-7470c8831596 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c214b2c97a7b41698b433eac8379442a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:56.638467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc4133-c741-45aa-b44a-7470c8831596 service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:56.638785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc4133-c741-45aa-b44a-7470c8831596 service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:56.639075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc4133-c741-45aa-b44a-7470c8831596 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:56.649169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc4133-c741-45aa-b44a-7470c8831596 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:56.650764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc4133-c741-45aa-b44a-7470c8831596 service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:56.660436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 924/1862] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:56.684839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-eb3b9a66-1231-4b03-99c4-25d7f25bd262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de47d1e999d4b4a927d9163649913a5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:56.685473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-eb3b9a66-1231-4b03-99c4-25d7f25bd262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de47d1e999d4b4a927d9163649913a5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:56.693814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 939/1863] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:05:56 2022] DELETE /identity/v3/users/5de47d1e999d4b4a927d9163649913a5 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:56.846062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:57.084426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:57.226843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc4133-c741-45aa-b44a-7470c8831596 service glance] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:57.238799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc4133-c741-45aa-b44a-7470c8831596 service glance] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:57.313241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:57.316232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:57.316232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:57.316232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] PATH_INFO: `/v3/projects/8ec1654d167244e89717093d25fbc78c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:57.323659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec1654d167244e89717093d25fbc78c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:57.325881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:57.461453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec1654d167244e89717093d25fbc78c', 'request_id': 'req-0d34b862-24ed-45b7-8bc7-36e91997c6d8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:57.468956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec1654d167244e89717093d25fbc78c', 'request_id': 'req-0d34b862-24ed-45b7-8bc7-36e91997c6d8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:57.480398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec1654d167244e89717093d25fbc78c', 'request_id': 'req-0d34b862-24ed-45b7-8bc7-36e91997c6d8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:57.492270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:57.560336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] The token cache is being invalidate because project 8ec1654d167244e89717093d25fbc78c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:05:57.560740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:57.561656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d34b862-24ed-45b7-8bc7-36e91997c6d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:05:57.569528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 925/1864] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:05:57 2022] DELETE /identity/v3/projects/8ec1654d167244e89717093d25fbc78c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:57.569779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd1e17-ed43-45aa-b975-56fd0a824c82 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:57.569779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd1e17-ed43-45aa-b975-56fd0a824c82 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:57.569953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd1e17-ed43-45aa-b975-56fd0a824c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:57.595413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-28cd1e17-ed43-45aa-b975-56fd0a824c82 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:57.596637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-28cd1e17-ed43-45aa-b975-56fd0a824c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:57.666511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 940/1865] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:57.675723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dd771-7610-4c57-a4a7-d36fc17c0a20 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:57.675985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dd771-7610-4c57-a4a7-d36fc17c0a20 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:57.676182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dd771-7610-4c57-a4a7-d36fc17c0a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:57.695959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-785dd771-7610-4c57-a4a7-d36fc17c0a20 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:57.697090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-785dd771-7610-4c57-a4a7-d36fc17c0a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:57.705221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 926/1866] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:57.710114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd1e17-ed43-45aa-b975-56fd0a824c82 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:57.719864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd1e17-ed43-45aa-b975-56fd0a824c82 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:57.725144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd42207-5990-48bf-9fc5-05d7c6e6cbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:57.725759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd42207-5990-48bf-9fc5-05d7c6e6cbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:57.726038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd42207-5990-48bf-9fc5-05d7c6e6cbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:57.726315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd42207-5990-48bf-9fc5-05d7c6e6cbe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:57.727002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd42207-5990-48bf-9fc5-05d7c6e6cbe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:05:57.727311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd42207-5990-48bf-9fc5-05d7c6e6cbe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:57.728986 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd42207-5990-48bf-9fc5-05d7c6e6cbe7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:57.744775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 941/1867] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:05:57.748992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dd771-7610-4c57-a4a7-d36fc17c0a20 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:57.758953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dd771-7610-4c57-a4a7-d36fc17c0a20 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:57.763790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96777-8a50-4929-8b50-19062b0748dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:57.764389 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96777-8a50-4929-8b50-19062b0748dc admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:57.764666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96777-8a50-4929-8b50-19062b0748dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:57.764936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96777-8a50-4929-8b50-19062b0748dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:57.765655 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c96777-8a50-4929-8b50-19062b0748dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:57.767631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c96777-8a50-4929-8b50-19062b0748dc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:57.825353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:57.910648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 927/1868] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:57 2022] POST /identity/v3/projects => generated 393 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:57.913440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd42207-5990-48bf-9fc5-05d7c6e6cbe7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:57.924363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd42207-5990-48bf-9fc5-05d7c6e6cbe7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:57.986560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350c3fa-576d-4469-aff2-a91c96ca6328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:57.987229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350c3fa-576d-4469-aff2-a91c96ca6328 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:57.987503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350c3fa-576d-4469-aff2-a91c96ca6328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:57.987773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350c3fa-576d-4469-aff2-a91c96ca6328 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:57.988407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350c3fa-576d-4469-aff2-a91c96ca6328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:57.990339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350c3fa-576d-4469-aff2-a91c96ca6328 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:58.122885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 942/1869] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:57 2022] POST /identity/v3/users => generated 297 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:05:58.127188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96777-8a50-4929-8b50-19062b0748dc admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:58.141765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96777-8a50-4929-8b50-19062b0748dc admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:58.153118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f9d2f-bf68-4da3-bada-9015dcfd79cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:58.153734 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f9d2f-bf68-4da3-bada-9015dcfd79cf admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:58.154012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f9d2f-bf68-4da3-bada-9015dcfd79cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:58.154282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f9d2f-bf68-4da3-bada-9015dcfd79cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:58.154966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f9d2f-bf68-4da3-bada-9015dcfd79cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:58.156558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f9d2f-bf68-4da3-bada-9015dcfd79cf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:58.166930 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 928/1870] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:05:58.170886 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350c3fa-576d-4469-aff2-a91c96ca6328 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:58.181513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350c3fa-576d-4469-aff2-a91c96ca6328 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:58.186352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766efb9-3f7c-4e57-ae2d-13e5c5a92e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:58.186984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766efb9-3f7c-4e57-ae2d-13e5c5a92e62 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:58.187271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766efb9-3f7c-4e57-ae2d-13e5c5a92e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:58.187545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766efb9-3f7c-4e57-ae2d-13e5c5a92e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:58.188355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766efb9-3f7c-4e57-ae2d-13e5c5a92e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:58.189968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766efb9-3f7c-4e57-ae2d-13e5c5a92e62 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:58.201487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 943/1871] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:58.205518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f9d2f-bf68-4da3-bada-9015dcfd79cf admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:58.215575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f9d2f-bf68-4da3-bada-9015dcfd79cf admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:58.220175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de3396-1945-4522-941f-1ecc87fde5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:58.220823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de3396-1945-4522-941f-1ecc87fde5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:58.221109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de3396-1945-4522-941f-1ecc87fde5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:58.221387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de3396-1945-4522-941f-1ecc87fde5dc admin admin] PATH_INFO: `/v3/projects/49036110a84640698c33039c00a6a1f9/users/607e34f56ce0433784257d921323e6bf/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:58.236305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de3396-1945-4522-941f-1ecc87fde5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=49036110a84640698c33039c00a6a1f9, user_id=607e34f56ce0433784257d921323e6bf, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:58.239033 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de3396-1945-4522-941f-1ecc87fde5dc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:58.304407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 929/1872] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:58 2022] PUT /identity/v3/projects/49036110a84640698c33039c00a6a1f9/users/607e34f56ce0433784257d921323e6bf/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:58.314583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef594de-4835-4b63-a308-b881afcd0001 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:58.314975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef594de-4835-4b63-a308-b881afcd0001 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:58.315207 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef594de-4835-4b63-a308-b881afcd0001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:58.346044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-bef594de-4835-4b63-a308-b881afcd0001 None None] MFA Rules not processed for user `607e34f56ce0433784257d921323e6bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:58.347588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bef594de-4835-4b63-a308-b881afcd0001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:58.448053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 944/1873] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:58.455075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de3396-1945-4522-941f-1ecc87fde5dc admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:58.521538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de3396-1945-4522-941f-1ecc87fde5dc admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:58.526941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ead440-3409-40aa-a31e-19c1db3650f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:58.527597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ead440-3409-40aa-a31e-19c1db3650f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:58.527878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ead440-3409-40aa-a31e-19c1db3650f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:58.528153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ead440-3409-40aa-a31e-19c1db3650f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:58.528823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ead440-3409-40aa-a31e-19c1db3650f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:58.530920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ead440-3409-40aa-a31e-19c1db3650f8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:58.625775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 930/1874] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:05:58 2022] POST /identity/v3/projects => generated 393 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:05:58.627246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef594de-4835-4b63-a308-b881afcd0001 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:58.638492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef594de-4835-4b63-a308-b881afcd0001 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:58.643251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:58.778050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:58.788406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b1612-86aa-475a-a687-6d79c4f0d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:58.789242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1612-86aa-475a-a687-6d79c4f0d030 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:58.792359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1612-86aa-475a-a687-6d79c4f0d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:58.792359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1612-86aa-475a-a687-6d79c4f0d030 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:58.792359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b1612-86aa-475a-a687-6d79c4f0d030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:58.792831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b1612-86aa-475a-a687-6d79c4f0d030 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:58.908120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 945/1875] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:05:58 2022] POST /identity/v3/users => generated 303 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:05:58.911163 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ead440-3409-40aa-a31e-19c1db3650f8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:58.922108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ead440-3409-40aa-a31e-19c1db3650f8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:58.926626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5defd1-f2a5-4413-b2e4-c311b3cd23a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:58.927558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5defd1-f2a5-4413-b2e4-c311b3cd23a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:58.927915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5defd1-f2a5-4413-b2e4-c311b3cd23a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:58.928370 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5defd1-f2a5-4413-b2e4-c311b3cd23a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:58.929387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5defd1-f2a5-4413-b2e4-c311b3cd23a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:58.931143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5defd1-f2a5-4413-b2e4-c311b3cd23a3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:58.943239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 931/1876] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:58.946427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b1612-86aa-475a-a687-6d79c4f0d030 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:58.957578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b1612-86aa-475a-a687-6d79c4f0d030 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:58.964714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0c25d-fcac-431f-83b4-00d93c988bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:58.965396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c25d-fcac-431f-83b4-00d93c988bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:58.965822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c25d-fcac-431f-83b4-00d93c988bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:58.966106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c25d-fcac-431f-83b4-00d93c988bb6 admin admin] PATH_INFO: `/v3/projects/5051d62df9b442479debc53c4e84eaac/users/86dca4ee3d6043e5808f334292c922ea/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:58.988087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0c25d-fcac-431f-83b4-00d93c988bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5051d62df9b442479debc53c4e84eaac, user_id=86dca4ee3d6043e5808f334292c922ea, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:58.991540 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0c25d-fcac-431f-83b4-00d93c988bb6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:59.108155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 946/1877] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:58 2022] PUT /identity/v3/projects/5051d62df9b442479debc53c4e84eaac/users/86dca4ee3d6043e5808f334292c922ea/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:05:59.112176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5defd1-f2a5-4413-b2e4-c311b3cd23a3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:59.122671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5defd1-f2a5-4413-b2e4-c311b3cd23a3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:59.208051 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218bd5d-e95c-4d9b-9449-d6d97a9f64a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:59.209319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218bd5d-e95c-4d9b-9449-d6d97a9f64a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:59.209604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218bd5d-e95c-4d9b-9449-d6d97a9f64a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:59.209881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218bd5d-e95c-4d9b-9449-d6d97a9f64a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:59.210533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218bd5d-e95c-4d9b-9449-d6d97a9f64a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:59.212247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218bd5d-e95c-4d9b-9449-d6d97a9f64a3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:59.222656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 932/1878] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:05:59.227573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0c25d-fcac-431f-83b4-00d93c988bb6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:59.238285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0c25d-fcac-431f-83b4-00d93c988bb6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:59.243318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87b111-885f-48c6-8c03-be043258dc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:59.244116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87b111-885f-48c6-8c03-be043258dc0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:59.244488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87b111-885f-48c6-8c03-be043258dc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:59.244982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87b111-885f-48c6-8c03-be043258dc0d admin admin] PATH_INFO: `/v3/projects/5051d62df9b442479debc53c4e84eaac/users/86dca4ee3d6043e5808f334292c922ea/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:59.247030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87b111-885f-48c6-8c03-be043258dc0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5051d62df9b442479debc53c4e84eaac, user_id=86dca4ee3d6043e5808f334292c922ea, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:59.249729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87b111-885f-48c6-8c03-be043258dc0d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:59.262030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 947/1879] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:05:59 2022] PUT /identity/v3/projects/5051d62df9b442479debc53c4e84eaac/users/86dca4ee3d6043e5808f334292c922ea/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:05:59.273662 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1899a-0248-4dcd-a5dd-3394f61a9b54 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:59.273951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1899a-0248-4dcd-a5dd-3394f61a9b54 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:59.274234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1899a-0248-4dcd-a5dd-3394f61a9b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:59.312855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ebf1899a-0248-4dcd-a5dd-3394f61a9b54 None None] MFA Rules not processed for user `86dca4ee3d6043e5808f334292c922ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:05:59.314617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ebf1899a-0248-4dcd-a5dd-3394f61a9b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:59.399359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 933/1880] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:05:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:05:59.410510 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87b111-885f-48c6-8c03-be043258dc0d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:59.412324 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1e87b111-885f-48c6-8c03-be043258dc0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:05:59.474782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:59.477021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87b111-885f-48c6-8c03-be043258dc0d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:59.480710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756c169-2714-4a10-a3db-e1ccc10d1630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:59.481322 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756c169-2714-4a10-a3db-e1ccc10d1630 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:59.481627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756c169-2714-4a10-a3db-e1ccc10d1630 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:59.481896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756c169-2714-4a10-a3db-e1ccc10d1630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:59.490904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756c169-2714-4a10-a3db-e1ccc10d1630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:59.492314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756c169-2714-4a10-a3db-e1ccc10d1630 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:59.501485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 948/1881] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:05:59.563927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1899a-0248-4dcd-a5dd-3394f61a9b54 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:05:59.575218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1899a-0248-4dcd-a5dd-3394f61a9b54 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:05:59.578840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c21f0c-702a-4e32-bc68-f395cbe92197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:05:59.579494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c21f0c-702a-4e32-bc68-f395cbe92197 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:05:59.579775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c21f0c-702a-4e32-bc68-f395cbe92197 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:05:59.580045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c21f0c-702a-4e32-bc68-f395cbe92197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:05:59.643673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c21f0c-702a-4e32-bc68-f395cbe92197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:05:59.645208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c21f0c-702a-4e32-bc68-f395cbe92197 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:05:59.651849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:05:59.656659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 934/1882] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:05:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:00.259916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:00.326904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:00.538045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:00.711328 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:01.223917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:01.437264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c5ce5-28ed-4a7c-aac0-eb2893a8ddc7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:01.437861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c5ce5-28ed-4a7c-aac0-eb2893a8ddc7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:01.437954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c5ce5-28ed-4a7c-aac0-eb2893a8ddc7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:01.439801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 949/1883] 10.222.0.59 () {54 vars in 921 bytes} [Fri Sep 16 08:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:06:01.461286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e698a9e-73e9-4a4b-82ca-0dbebc0da4af None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:01.461818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e698a9e-73e9-4a4b-82ca-0dbebc0da4af None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:01.462300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e698a9e-73e9-4a4b-82ca-0dbebc0da4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:01.500152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8e698a9e-73e9-4a4b-82ca-0dbebc0da4af None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:01.501617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8e698a9e-73e9-4a4b-82ca-0dbebc0da4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:01.587619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 935/1884] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Sep 16 08:06:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:01.705518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:01.735203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c5ce5-28ed-4a7c-aac0-eb2893a8ddc7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:01.737439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e54c5ce5-28ed-4a7c-aac0-eb2893a8ddc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:01.757877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c5ce5-28ed-4a7c-aac0-eb2893a8ddc7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:01.763635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd7ccd-c9d8-4af6-94b8-f94737bcced4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:01.764525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd7ccd-c9d8-4af6-94b8-f94737bcced4 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:01.764864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd7ccd-c9d8-4af6-94b8-f94737bcced4 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:01.765204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd7ccd-c9d8-4af6-94b8-f94737bcced4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:01.784236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd7ccd-c9d8-4af6-94b8-f94737bcced4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:01.784236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd7ccd-c9d8-4af6-94b8-f94737bcced4 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:01.803857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 950/1885] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:02.224547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e698a9e-73e9-4a4b-82ca-0dbebc0da4af None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:02.228217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8e698a9e-73e9-4a4b-82ca-0dbebc0da4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:02.330980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e698a9e-73e9-4a4b-82ca-0dbebc0da4af None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:02.347660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:02.348352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:02.348632 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:02.348949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] PATH_INFO: `/v3/users/943da8c557ae4bcc9451e0eb290744f9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:02.350326 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=943da8c557ae4bcc9451e0eb290744f9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:02.352348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:02.595906 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '943da8c557ae4bcc9451e0eb290744f9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:02.620463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '943da8c557ae4bcc9451e0eb290744f9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:02.629936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '943da8c557ae4bcc9451e0eb290744f9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:02.706938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '943da8c557ae4bcc9451e0eb290744f9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:02.717277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '943da8c557ae4bcc9451e0eb290744f9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:02.730992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 936/1886] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:02 2022] DELETE /identity/v3/users/943da8c557ae4bcc9451e0eb290744f9 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:02.740034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776e609-9fb7-4883-a773-5b059e3d3b62 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:02.741445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776e609-9fb7-4883-a773-5b059e3d3b62 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:02.741907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776e609-9fb7-4883-a773-5b059e3d3b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:02.784213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1776e609-9fb7-4883-a773-5b059e3d3b62 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:02.785738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1776e609-9fb7-4883-a773-5b059e3d3b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:02.881578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 951/1887] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:02.891919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:02.893113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:02.894633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:02.971903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c28cc-1487-4925-8fda-a272ce1eb047 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:02.979240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68513fb4-6a73-48cb-90dd-c48f3f1447c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:02.979936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68513fb4-6a73-48cb-90dd-c48f3f1447c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:02.980286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68513fb4-6a73-48cb-90dd-c48f3f1447c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:02.980643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68513fb4-6a73-48cb-90dd-c48f3f1447c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:02.991686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68513fb4-6a73-48cb-90dd-c48f3f1447c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:02.993127 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68513fb4-6a73-48cb-90dd-c48f3f1447c2 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:03.009626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 937/1888] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:03.550834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776e609-9fb7-4883-a773-5b059e3d3b62 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:03.555759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1776e609-9fb7-4883-a773-5b059e3d3b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:03.582846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776e609-9fb7-4883-a773-5b059e3d3b62 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:03.601150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:03.601885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:03.602174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:03.603704 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] PATH_INFO: `/v3/projects/c634aa313ef04e1eb8b703542887fcd5` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:03.607227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c634aa313ef04e1eb8b703542887fcd5)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:03.609659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:03.638810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:03.769328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c634aa313ef04e1eb8b703542887fcd5', 'request_id': 'req-6b16fa01-699f-471c-8353-25c37ee1468a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:03.778317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c634aa313ef04e1eb8b703542887fcd5', 'request_id': 'req-6b16fa01-699f-471c-8353-25c37ee1468a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:03.787655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c634aa313ef04e1eb8b703542887fcd5', 'request_id': 'req-6b16fa01-699f-471c-8353-25c37ee1468a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:03.898387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] The token cache is being invalidate because project c634aa313ef04e1eb8b703542887fcd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:03.898944 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:03.899904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:03.901762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 952/1889] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:03 2022] DELETE /identity/v3/projects/c634aa313ef04e1eb8b703542887fcd5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:03.906313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68513fb4-6a73-48cb-90dd-c48f3f1447c2 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:03.916924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:03.932539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68513fb4-6a73-48cb-90dd-c48f3f1447c2 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:04.024592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:04.025771 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:04.026131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:04.026519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] PATH_INFO: `/v3/users/711140ebf89242dba466dd868b74707a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:04.027790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] RBAC: Authorizing `identity:delete_user(user_id=711140ebf89242dba466dd868b74707a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:04.029899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:04.313496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '711140ebf89242dba466dd868b74707a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:04.336532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '711140ebf89242dba466dd868b74707a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:04.345208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '711140ebf89242dba466dd868b74707a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:04.413556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '711140ebf89242dba466dd868b74707a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:04.414168 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '711140ebf89242dba466dd868b74707a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:04.420598 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 938/1890] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:03 2022] DELETE /identity/v3/users/711140ebf89242dba466dd868b74707a => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:05.140711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:05.143477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:05.161108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16fa01-699f-471c-8353-25c37ee1468a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:05.232215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:05.232919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:05.233119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:05.233390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] PATH_INFO: `/v3/projects/931f90a116d740ef9288b4333a7bdcd4` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:05.234596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] RBAC: Authorizing `identity:delete_project(project_id=931f90a116d740ef9288b4333a7bdcd4)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:05.236573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:05.350854 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '931f90a116d740ef9288b4333a7bdcd4', 'request_id': 'req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:05.357251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '931f90a116d740ef9288b4333a7bdcd4', 'request_id': 'req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:05.363141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '931f90a116d740ef9288b4333a7bdcd4', 'request_id': 'req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:05.450939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:05.455985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:05.456162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] The token cache is being invalidate because project 931f90a116d740ef9288b4333a7bdcd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:05.456162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:05.456162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4b920b4-3c02-4baf-b2a1-8c841bcd72de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:05.459689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 953/1891] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:05 2022] DELETE /identity/v3/projects/931f90a116d740ef9288b4333a7bdcd4 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:05.464780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a4e2e-f97e-4bee-b4cf-18bc8c9eff5f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:05.464780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a4e2e-f97e-4bee-b4cf-18bc8c9eff5f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:05.465015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a4e2e-f97e-4bee-b4cf-18bc8c9eff5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:05.471114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a32e2-c3d2-4e2e-a8a9-ffc5d2b8deac admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:05.487341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-698a4e2e-f97e-4bee-b4cf-18bc8c9eff5f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:05.488559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-698a4e2e-f97e-4bee-b4cf-18bc8c9eff5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:05.499563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:05.531639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:05.541843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c4c20-57e7-4980-8c39-966a3256b2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:05.542880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c4c20-57e7-4980-8c39-966a3256b2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:05.543170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c4c20-57e7-4980-8c39-966a3256b2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:05.543444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c4c20-57e7-4980-8c39-966a3256b2e0 admin admin] PATH_INFO: `/v3/users/607e34f56ce0433784257d921323e6bf` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:05.545262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c4c20-57e7-4980-8c39-966a3256b2e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=607e34f56ce0433784257d921323e6bf)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:05.547285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c4c20-57e7-4980-8c39-966a3256b2e0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:05.561903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 954/1892] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:05.570244 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687be75-a8a5-49d7-bd6e-6636f25dc485 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:05.570640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687be75-a8a5-49d7-bd6e-6636f25dc485 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:05.570947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687be75-a8a5-49d7-bd6e-6636f25dc485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:05.590391 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b687be75-a8a5-49d7-bd6e-6636f25dc485 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:05.591628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b687be75-a8a5-49d7-bd6e-6636f25dc485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:05.599661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 955/1893] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:05.603976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687be75-a8a5-49d7-bd6e-6636f25dc485 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:05.613805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687be75-a8a5-49d7-bd6e-6636f25dc485 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:05.619318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671158c1-4b4e-4191-8e3b-8bd262ea51f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:05.622278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671158c1-4b4e-4191-8e3b-8bd262ea51f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:05.622278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671158c1-4b4e-4191-8e3b-8bd262ea51f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:05.622278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671158c1-4b4e-4191-8e3b-8bd262ea51f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:05.622278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671158c1-4b4e-4191-8e3b-8bd262ea51f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:05.622278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671158c1-4b4e-4191-8e3b-8bd262ea51f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:05.623087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671158c1-4b4e-4191-8e3b-8bd262ea51f1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:05.638329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 956/1894] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:06:05.642792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671158c1-4b4e-4191-8e3b-8bd262ea51f1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:05.653822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671158c1-4b4e-4191-8e3b-8bd262ea51f1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:05.658860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc1e28-faa8-4be4-8c19-bd0caf62ad8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:05.659558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc1e28-faa8-4be4-8c19-bd0caf62ad8f admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:05.659840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc1e28-faa8-4be4-8c19-bd0caf62ad8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:05.660371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc1e28-faa8-4be4-8c19-bd0caf62ad8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:05.661071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc1e28-faa8-4be4-8c19-bd0caf62ad8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:05.663164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc1e28-faa8-4be4-8c19-bd0caf62ad8f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:05.750176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ce4c4c20-57e7-4980-8c39-966a3256b2e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607e34f56ce0433784257d921323e6bf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:05.766585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 957/1895] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:05 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:05.767916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc1e28-faa8-4be4-8c19-bd0caf62ad8f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:05.768754 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ce4c4c20-57e7-4980-8c39-966a3256b2e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607e34f56ce0433784257d921323e6bf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:05.776336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ce4c4c20-57e7-4980-8c39-966a3256b2e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607e34f56ce0433784257d921323e6bf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:05.832679 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc1e28-faa8-4be4-8c19-bd0caf62ad8f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:05.837496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77c3fe-e100-4be5-91ce-560dab0b6b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:05.838116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77c3fe-e100-4be5-91ce-560dab0b6b51 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:05.838398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77c3fe-e100-4be5-91ce-560dab0b6b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:05.838670 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77c3fe-e100-4be5-91ce-560dab0b6b51 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:05.839359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77c3fe-e100-4be5-91ce-560dab0b6b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:05.843627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77c3fe-e100-4be5-91ce-560dab0b6b51 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:05.862318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ce4c4c20-57e7-4980-8c39-966a3256b2e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607e34f56ce0433784257d921323e6bf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:05.862956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ce4c4c20-57e7-4980-8c39-966a3256b2e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607e34f56ce0433784257d921323e6bf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:05.869785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 939/1896] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:05 2022] DELETE /identity/v3/users/607e34f56ce0433784257d921323e6bf => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:05.874057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198cfc4-7ce7-4eb1-97f8-68300e3df40b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:05.874336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198cfc4-7ce7-4eb1-97f8-68300e3df40b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:05.874605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198cfc4-7ce7-4eb1-97f8-68300e3df40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:05.885943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:05.897468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7198cfc4-7ce7-4eb1-97f8-68300e3df40b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:05.898836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7198cfc4-7ce7-4eb1-97f8-68300e3df40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:05.949389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 958/1897] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:05 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:05.953479 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77c3fe-e100-4be5-91ce-560dab0b6b51 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:05.955865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0d77c3fe-e100-4be5-91ce-560dab0b6b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:05.967724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 940/1898] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:05.971053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77c3fe-e100-4be5-91ce-560dab0b6b51 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:05.982332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d4b8a-ad36-455e-8d3b-f4eb4359c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:05.982635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198cfc4-7ce7-4eb1-97f8-68300e3df40b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:05.982987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d4b8a-ad36-455e-8d3b-f4eb4359c3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:05.983271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d4b8a-ad36-455e-8d3b-f4eb4359c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:05.983651 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d4b8a-ad36-455e-8d3b-f4eb4359c3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:05.984191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d4b8a-ad36-455e-8d3b-f4eb4359c3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:05.984349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7198cfc4-7ce7-4eb1-97f8-68300e3df40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:05.985847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d4b8a-ad36-455e-8d3b-f4eb4359c3ad admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:05.998079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 959/1899] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:06.005649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d4b8a-ad36-455e-8d3b-f4eb4359c3ad admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:06.017774 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d4b8a-ad36-455e-8d3b-f4eb4359c3ad admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:06.023780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fc804-d432-4d86-a5a9-dffc0d18c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:06.027400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fc804-d432-4d86-a5a9-dffc0d18c174 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:06.027400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fc804-d432-4d86-a5a9-dffc0d18c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:06.027400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fc804-d432-4d86-a5a9-dffc0d18c174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:06.027400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fc804-d432-4d86-a5a9-dffc0d18c174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:06.027400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fc804-d432-4d86-a5a9-dffc0d18c174 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:06.041030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 960/1900] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:06.045015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198cfc4-7ce7-4eb1-97f8-68300e3df40b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:06.045224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fc804-d432-4d86-a5a9-dffc0d18c174 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:06.048905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f173d-7806-4b25-a88e-07fc972e79e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:06.049512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f173d-7806-4b25-a88e-07fc972e79e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:06.049787 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f173d-7806-4b25-a88e-07fc972e79e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:06.050082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f173d-7806-4b25-a88e-07fc972e79e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:06.060646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fc804-d432-4d86-a5a9-dffc0d18c174 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:06.060824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f173d-7806-4b25-a88e-07fc972e79e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:06.062150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f173d-7806-4b25-a88e-07fc972e79e7 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:06.065405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89788c3-1458-4cfd-8446-019158f68546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:06.066071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89788c3-1458-4cfd-8446-019158f68546 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:06.066350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89788c3-1458-4cfd-8446-019158f68546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:06.066624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89788c3-1458-4cfd-8446-019158f68546 admin admin] PATH_INFO: `/v3/projects/f2969e7a601449e4bdba93bdfb5cc89d/users/c4554202a87c4d24ae92a26ffebdd06f/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:06.073571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 941/1901] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:06.084005 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89788c3-1458-4cfd-8446-019158f68546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2969e7a601449e4bdba93bdfb5cc89d, user_id=c4554202a87c4d24ae92a26ffebdd06f, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:06.087374 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89788c3-1458-4cfd-8446-019158f68546 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:06.192621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 961/1902] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:06 2022] PUT /identity/v3/projects/f2969e7a601449e4bdba93bdfb5cc89d/users/c4554202a87c4d24ae92a26ffebdd06f/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:06.204997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a9a1d-e556-430e-a843-46eeadad1527 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:06.205732 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a9a1d-e556-430e-a843-46eeadad1527 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:06.205732 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a9a1d-e556-430e-a843-46eeadad1527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:06.237571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-838a9a1d-e556-430e-a843-46eeadad1527 None None] MFA Rules not processed for user `c4554202a87c4d24ae92a26ffebdd06f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:06.238972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-838a9a1d-e556-430e-a843-46eeadad1527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:06.354790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 942/1903] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:06.359019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89788c3-1458-4cfd-8446-019158f68546 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:06.472799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89788c3-1458-4cfd-8446-019158f68546 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:06.480739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e18fc1-4c2f-4df6-824f-7d2458096993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:06.481706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e18fc1-4c2f-4df6-824f-7d2458096993 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:06.482098 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e18fc1-4c2f-4df6-824f-7d2458096993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:06.482600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e18fc1-4c2f-4df6-824f-7d2458096993 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:06.484024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e18fc1-4c2f-4df6-824f-7d2458096993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:06.488525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e18fc1-4c2f-4df6-824f-7d2458096993 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:06.622061 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 962/1904] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:06 2022] POST /identity/v3/projects => generated 381 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:06.626930 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a9a1d-e556-430e-a843-46eeadad1527 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:06.662380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a9a1d-e556-430e-a843-46eeadad1527 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:06.752903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989e47b-ac09-417d-8830-8ec929340627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:06.753753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989e47b-ac09-417d-8830-8ec929340627 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:06.754041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989e47b-ac09-417d-8830-8ec929340627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:06.754319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989e47b-ac09-417d-8830-8ec929340627 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:06.755519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989e47b-ac09-417d-8830-8ec929340627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:06.757510 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989e47b-ac09-417d-8830-8ec929340627 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:06.796730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:06.804921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e18fc1-4c2f-4df6-824f-7d2458096993 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:06.813982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-62e18fc1-4c2f-4df6-824f-7d2458096993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:06.825028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e18fc1-4c2f-4df6-824f-7d2458096993 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:06.836538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:06.837208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:06.837838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:06.838047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] PATH_INFO: `/v3/projects/49036110a84640698c33039c00a6a1f9` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:06.839532 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=49036110a84640698c33039c00a6a1f9)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:06.841646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:06.884333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 943/1905] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:06 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:06.888225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989e47b-ac09-417d-8830-8ec929340627 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:06.898222 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989e47b-ac09-417d-8830-8ec929340627 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:06.910806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a06757-2671-4a6c-a173-71ec97d1e395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:06.912003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a06757-2671-4a6c-a173-71ec97d1e395 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:06.913546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a06757-2671-4a6c-a173-71ec97d1e395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:06.914744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a06757-2671-4a6c-a173-71ec97d1e395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:06.914744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a06757-2671-4a6c-a173-71ec97d1e395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:06.916203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a06757-2671-4a6c-a173-71ec97d1e395 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:06.930963 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 944/1906] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:06.933485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a06757-2671-4a6c-a173-71ec97d1e395 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:06.948436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a06757-2671-4a6c-a173-71ec97d1e395 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:06.950243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49036110a84640698c33039c00a6a1f9', 'request_id': 'req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:06.971840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49036110a84640698c33039c00a6a1f9', 'request_id': 'req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:06.980815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49036110a84640698c33039c00a6a1f9', 'request_id': 'req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:07.038286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61cd54-7916-4ff4-a24e-68cfabf02392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:07.039516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61cd54-7916-4ff4-a24e-68cfabf02392 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:07.039903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61cd54-7916-4ff4-a24e-68cfabf02392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:07.040282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61cd54-7916-4ff4-a24e-68cfabf02392 admin admin] PATH_INFO: `/v3/projects/e8684766e3524f50875c0bb98ad491bb/users/81690cbbf33347798d72186e3075a117/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:07.059032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61cd54-7916-4ff4-a24e-68cfabf02392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8684766e3524f50875c0bb98ad491bb, user_id=81690cbbf33347798d72186e3075a117, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:07.062086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61cd54-7916-4ff4-a24e-68cfabf02392 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:07.146086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:07.159604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 945/1907] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:06 2022] PUT /identity/v3/projects/e8684766e3524f50875c0bb98ad491bb/users/81690cbbf33347798d72186e3075a117/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:07.161806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] The token cache is being invalidate because project 49036110a84640698c33039c00a6a1f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:07.162958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:07.163674 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61cd54-7916-4ff4-a24e-68cfabf02392 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:07.164245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:07.165859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8a61cd54-7916-4ff4-a24e-68cfabf02392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:07.166781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 963/1908] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:06 2022] DELETE /identity/v3/projects/49036110a84640698c33039c00a6a1f9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:07.170912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:07.172862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:07.186235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439b90f-6ff8-4721-8aed-7d7b4dce80bc admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:07.245414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61cd54-7916-4ff4-a24e-68cfabf02392 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:07.252347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:07.253063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:07.253367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdfb59-ef1a-4847-951d-a82ecd52bfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:07.253531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:07.253817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] PATH_INFO: `/v3/users/86dca4ee3d6043e5808f334292c922ea` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:07.254557 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdfb59-ef1a-4847-951d-a82ecd52bfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:07.254901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdfb59-ef1a-4847-951d-a82ecd52bfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:07.255010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86dca4ee3d6043e5808f334292c922ea)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:07.255179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdfb59-ef1a-4847-951d-a82ecd52bfc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:07.255835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccdfb59-ef1a-4847-951d-a82ecd52bfc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:07.256950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:07.257741 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccdfb59-ef1a-4847-951d-a82ecd52bfc7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:07.279815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 946/1909] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:07.291743 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdfb59-ef1a-4847-951d-a82ecd52bfc7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:07.312724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdfb59-ef1a-4847-951d-a82ecd52bfc7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:07.325578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979dcdf-2b41-4bd3-9eb8-a1c77ce177ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:07.326501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979dcdf-2b41-4bd3-9eb8-a1c77ce177ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:07.326835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979dcdf-2b41-4bd3-9eb8-a1c77ce177ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:07.327155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979dcdf-2b41-4bd3-9eb8-a1c77ce177ac admin admin] PATH_INFO: `/v3/projects/e8684766e3524f50875c0bb98ad491bb/users/81690cbbf33347798d72186e3075a117/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:07.330872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979dcdf-2b41-4bd3-9eb8-a1c77ce177ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8684766e3524f50875c0bb98ad491bb, user_id=81690cbbf33347798d72186e3075a117, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:07.334345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979dcdf-2b41-4bd3-9eb8-a1c77ce177ac admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:07.358252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 947/1910] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:07 2022] PUT /identity/v3/projects/e8684766e3524f50875c0bb98ad491bb/users/81690cbbf33347798d72186e3075a117/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:07.369760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a165f4a-7229-463f-af73-619000f52d88 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:07.370131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a165f4a-7229-463f-af73-619000f52d88 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:07.370440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a165f4a-7229-463f-af73-619000f52d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:07.401661 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6a165f4a-7229-463f-af73-619000f52d88 None None] MFA Rules not processed for user `81690cbbf33347798d72186e3075a117`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:07.402894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6a165f4a-7229-463f-af73-619000f52d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:07.480502 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 948/1911] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:07.493959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a165f4a-7229-463f-af73-619000f52d88 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:07.496416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6a165f4a-7229-463f-af73-619000f52d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:07.517351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86dca4ee3d6043e5808f334292c922ea'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:07.533333 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86dca4ee3d6043e5808f334292c922ea'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:07.541033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86dca4ee3d6043e5808f334292c922ea'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:07.567864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a165f4a-7229-463f-af73-619000f52d88 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:07.567864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff97ea-1d43-4b56-a4c2-5cf1b4d47ae2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:07.568097 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff97ea-1d43-4b56-a4c2-5cf1b4d47ae2 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:07.568331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff97ea-1d43-4b56-a4c2-5cf1b4d47ae2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:07.568604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff97ea-1d43-4b56-a4c2-5cf1b4d47ae2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:07.580411 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff97ea-1d43-4b56-a4c2-5cf1b4d47ae2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:07.580676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff97ea-1d43-4b56-a4c2-5cf1b4d47ae2 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:07.592588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 949/1912] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:07.634931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86dca4ee3d6043e5808f334292c922ea'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:07.636905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:07.637859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86dca4ee3d6043e5808f334292c922ea'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:07.654993 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 964/1913] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:07 2022] DELETE /identity/v3/users/86dca4ee3d6043e5808f334292c922ea => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:07.755222 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff97ea-1d43-4b56-a4c2-5cf1b4d47ae2 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:07.756713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e7ff97ea-1d43-4b56-a4c2-5cf1b4d47ae2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:07.768076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff97ea-1d43-4b56-a4c2-5cf1b4d47ae2 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:07.825049 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0d8e-b816-476c-a06c-b0946e8472cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:07.825664 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0d8e-b816-476c-a06c-b0946e8472cf service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:07.825948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0d8e-b816-476c-a06c-b0946e8472cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:07.826224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0d8e-b816-476c-a06c-b0946e8472cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:07.828157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-442e0d8e-b816-476c-a06c-b0946e8472cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:07.897592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e0d8e-b816-476c-a06c-b0946e8472cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:07.900652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e0d8e-b816-476c-a06c-b0946e8472cf service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:07.911145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 950/1914] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:08.134181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:08.134181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:08.207602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda2dde-d0f7-4e4b-b04c-b7ff76cad191 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:08.212992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:08.213722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:08.214009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:08.214270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] PATH_INFO: `/v3/projects/5051d62df9b442479debc53c4e84eaac` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:08.215410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5051d62df9b442479debc53c4e84eaac)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:08.217343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:08.323473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5051d62df9b442479debc53c4e84eaac', 'request_id': 'req-ead3b31c-c266-4307-8be1-308e0d21d303'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:08.328598 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:08.330623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5051d62df9b442479debc53c4e84eaac', 'request_id': 'req-ead3b31c-c266-4307-8be1-308e0d21d303'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:08.337110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5051d62df9b442479debc53c4e84eaac', 'request_id': 'req-ead3b31c-c266-4307-8be1-308e0d21d303'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:08.450320 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0d8e-b816-476c-a06c-b0946e8472cf service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:08.452056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-442e0d8e-b816-476c-a06c-b0946e8472cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:08.472735 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] The token cache is being invalidate because project 5051d62df9b442479debc53c4e84eaac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:08.478974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:08.478974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ead3b31c-c266-4307-8be1-308e0d21d303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:08.479165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0d8e-b816-476c-a06c-b0946e8472cf service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:08.479273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 965/1915] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:08 2022] DELETE /identity/v3/projects/5051d62df9b442479debc53c4e84eaac => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:08.483940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9323321-c87a-4116-9e6c-6aafed3e0cab None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:08.484637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9323321-c87a-4116-9e6c-6aafed3e0cab None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:08.484914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9323321-c87a-4116-9e6c-6aafed3e0cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:08.507755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d9323321-c87a-4116-9e6c-6aafed3e0cab None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:08.508708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d9323321-c87a-4116-9e6c-6aafed3e0cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:08.570921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5a70f-ab40-4117-928f-f7afa4cf989f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:08.571571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5a70f-ab40-4117-928f-f7afa4cf989f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:08.571851 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5a70f-ab40-4117-928f-f7afa4cf989f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:08.572130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5a70f-ab40-4117-928f-f7afa4cf989f admin admin] PATH_INFO: `/v3/users/c4554202a87c4d24ae92a26ffebdd06f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:08.573358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5a70f-ab40-4117-928f-f7afa4cf989f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4554202a87c4d24ae92a26ffebdd06f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:08.575442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5a70f-ab40-4117-928f-f7afa4cf989f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:08.579167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 966/1916] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:08.588347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc225046-df11-4d19-b952-dd1fbe75eae4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:08.588736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc225046-df11-4d19-b952-dd1fbe75eae4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:08.589108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc225046-df11-4d19-b952-dd1fbe75eae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:08.609742 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cc225046-df11-4d19-b952-dd1fbe75eae4 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:08.611223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cc225046-df11-4d19-b952-dd1fbe75eae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:08.620714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 967/1917] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:08.624992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc225046-df11-4d19-b952-dd1fbe75eae4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:08.634782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc225046-df11-4d19-b952-dd1fbe75eae4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:08.639315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85af2b-a28d-458b-9bcd-e35dad24bbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:08.639917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85af2b-a28d-458b-9bcd-e35dad24bbae admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:08.640190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85af2b-a28d-458b-9bcd-e35dad24bbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:08.640460 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85af2b-a28d-458b-9bcd-e35dad24bbae admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:08.641108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85af2b-a28d-458b-9bcd-e35dad24bbae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:08.641404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85af2b-a28d-458b-9bcd-e35dad24bbae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:08.643019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85af2b-a28d-458b-9bcd-e35dad24bbae admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:08.658226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 968/1918] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:06:08.659228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85af2b-a28d-458b-9bcd-e35dad24bbae admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:08.672256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85af2b-a28d-458b-9bcd-e35dad24bbae admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:08.678520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4af6e0-54b0-4536-82c8-8763713f1c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:08.679158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4af6e0-54b0-4536-82c8-8763713f1c61 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:08.679436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4af6e0-54b0-4536-82c8-8763713f1c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:08.679707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4af6e0-54b0-4536-82c8-8763713f1c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:08.680345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4af6e0-54b0-4536-82c8-8763713f1c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:08.682348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4af6e0-54b0-4536-82c8-8763713f1c61 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:08.794636 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-beb5a70f-ab40-4117-928f-f7afa4cf989f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4554202a87c4d24ae92a26ffebdd06f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:08.809306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-beb5a70f-ab40-4117-928f-f7afa4cf989f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4554202a87c4d24ae92a26ffebdd06f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:08.813919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 969/1919] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:08 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:08.815178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4af6e0-54b0-4536-82c8-8763713f1c61 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:08.821180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-beb5a70f-ab40-4117-928f-f7afa4cf989f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4554202a87c4d24ae92a26ffebdd06f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:08.882125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4af6e0-54b0-4536-82c8-8763713f1c61 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:08.887410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23d165-8e6c-441f-8eb5-866be16f0ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:08.888107 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23d165-8e6c-441f-8eb5-866be16f0ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:08.897534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23d165-8e6c-441f-8eb5-866be16f0ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:08.897534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23d165-8e6c-441f-8eb5-866be16f0ba4 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:08.897534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23d165-8e6c-441f-8eb5-866be16f0ba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:08.897534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23d165-8e6c-441f-8eb5-866be16f0ba4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:08.904114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-beb5a70f-ab40-4117-928f-f7afa4cf989f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4554202a87c4d24ae92a26ffebdd06f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:08.904871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-beb5a70f-ab40-4117-928f-f7afa4cf989f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4554202a87c4d24ae92a26ffebdd06f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:08.914456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 951/1920] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:08 2022] DELETE /identity/v3/users/c4554202a87c4d24ae92a26ffebdd06f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:08.919105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87bbb2-02d8-4499-8c7a-85a09268d64d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:08.919495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87bbb2-02d8-4499-8c7a-85a09268d64d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:08.919987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87bbb2-02d8-4499-8c7a-85a09268d64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:08.943511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2e87bbb2-02d8-4499-8c7a-85a09268d64d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:08.945762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2e87bbb2-02d8-4499-8c7a-85a09268d64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:08.980712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 970/1921] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:08 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:08.985012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23d165-8e6c-441f-8eb5-866be16f0ba4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:08.986908 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-db23d165-8e6c-441f-8eb5-866be16f0ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:09.000861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23d165-8e6c-441f-8eb5-866be16f0ba4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:09.018704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 952/1922] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:09.030608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87bbb2-02d8-4499-8c7a-85a09268d64d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:09.032192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2e87bbb2-02d8-4499-8c7a-85a09268d64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:09.042267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:09.074581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef716346-fe59-435a-adaf-9d88c1aa3862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:09.075346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef716346-fe59-435a-adaf-9d88c1aa3862 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:09.075655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef716346-fe59-435a-adaf-9d88c1aa3862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:09.075957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef716346-fe59-435a-adaf-9d88c1aa3862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:09.076640 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef716346-fe59-435a-adaf-9d88c1aa3862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:09.078304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef716346-fe59-435a-adaf-9d88c1aa3862 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:09.090179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 971/1923] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:09.093976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef716346-fe59-435a-adaf-9d88c1aa3862 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:09.102457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87bbb2-02d8-4499-8c7a-85a09268d64d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:09.105889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef716346-fe59-435a-adaf-9d88c1aa3862 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:09.106031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca23ca-380d-4393-982d-fa88f26a0d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:09.110179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca23ca-380d-4393-982d-fa88f26a0d50 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:09.110179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca23ca-380d-4393-982d-fa88f26a0d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:09.110179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca23ca-380d-4393-982d-fa88f26a0d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:09.111859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93721-577d-4c71-8ea7-770be37cb018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:09.112617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93721-577d-4c71-8ea7-770be37cb018 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:09.113002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93721-577d-4c71-8ea7-770be37cb018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:09.113381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93721-577d-4c71-8ea7-770be37cb018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:09.114194 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba93721-577d-4c71-8ea7-770be37cb018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:09.116097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba93721-577d-4c71-8ea7-770be37cb018 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:09.117044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca23ca-380d-4393-982d-fa88f26a0d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:09.118856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca23ca-380d-4393-982d-fa88f26a0d50 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:09.127354 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 972/1924] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:09.128797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 953/1925] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:09.132551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93721-577d-4c71-8ea7-770be37cb018 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:09.143100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93721-577d-4c71-8ea7-770be37cb018 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:09.148762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bc398-0a1c-46cf-bfb0-2f761c4c9648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:09.149653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bc398-0a1c-46cf-bfb0-2f761c4c9648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:09.150010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bc398-0a1c-46cf-bfb0-2f761c4c9648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:09.150356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bc398-0a1c-46cf-bfb0-2f761c4c9648 admin admin] PATH_INFO: `/v3/projects/964cf92762cd4c8fa1d4ccabcae89fc2/users/021d0fca0f4d4f17bed4d6c209af54ea/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:09.167775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516bc398-0a1c-46cf-bfb0-2f761c4c9648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=964cf92762cd4c8fa1d4ccabcae89fc2, user_id=021d0fca0f4d4f17bed4d6c209af54ea, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:09.170536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516bc398-0a1c-46cf-bfb0-2f761c4c9648 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:09.262805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 973/1926] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:09 2022] PUT /identity/v3/projects/964cf92762cd4c8fa1d4ccabcae89fc2/users/021d0fca0f4d4f17bed4d6c209af54ea/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:09.273795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71aad1-25a2-4c07-88c4-926398de7e74 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:09.274076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71aad1-25a2-4c07-88c4-926398de7e74 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:09.274345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71aad1-25a2-4c07-88c4-926398de7e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:09.303382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1e71aad1-25a2-4c07-88c4-926398de7e74 None None] MFA Rules not processed for user `021d0fca0f4d4f17bed4d6c209af54ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:09.304595 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1e71aad1-25a2-4c07-88c4-926398de7e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:09.389861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 954/1927] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:09.394340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bc398-0a1c-46cf-bfb0-2f761c4c9648 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:09.437344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:09.485032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bc398-0a1c-46cf-bfb0-2f761c4c9648 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:09.493909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696a9c3-ecfd-45c2-926f-9ccf891879a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:09.497614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696a9c3-ecfd-45c2-926f-9ccf891879a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:09.497614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696a9c3-ecfd-45c2-926f-9ccf891879a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:09.497614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696a9c3-ecfd-45c2-926f-9ccf891879a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:09.497614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696a9c3-ecfd-45c2-926f-9ccf891879a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:09.497903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696a9c3-ecfd-45c2-926f-9ccf891879a0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:09.552685 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71aad1-25a2-4c07-88c4-926398de7e74 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:09.554241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1e71aad1-25a2-4c07-88c4-926398de7e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:09.573149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71aad1-25a2-4c07-88c4-926398de7e74 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:09.590302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 974/1928] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:09 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:09.590442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:09.591006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:09.591286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:09.591568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] PATH_INFO: `/v3/projects/f2969e7a601449e4bdba93bdfb5cc89d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:09.591695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696a9c3-ecfd-45c2-926f-9ccf891879a0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:09.593586 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2969e7a601449e4bdba93bdfb5cc89d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:09.598430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:09.696173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696a9c3-ecfd-45c2-926f-9ccf891879a0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:09.701790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b184be-bcf2-4331-ac1e-1cf4b6ca51f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:09.707928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b184be-bcf2-4331-ac1e-1cf4b6ca51f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:09.707928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b184be-bcf2-4331-ac1e-1cf4b6ca51f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:09.707928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b184be-bcf2-4331-ac1e-1cf4b6ca51f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:09.707928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b184be-bcf2-4331-ac1e-1cf4b6ca51f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:09.707928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b184be-bcf2-4331-ac1e-1cf4b6ca51f2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:09.729339 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2969e7a601449e4bdba93bdfb5cc89d', 'request_id': 'req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:09.744949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2969e7a601449e4bdba93bdfb5cc89d', 'request_id': 'req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:09.758778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2969e7a601449e4bdba93bdfb5cc89d', 'request_id': 'req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:09.797162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 975/1929] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:09 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:09.798475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b184be-bcf2-4331-ac1e-1cf4b6ca51f2 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:09.806623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:09.810403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b184be-bcf2-4331-ac1e-1cf4b6ca51f2 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:09.852505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] The token cache is being invalidate because project f2969e7a601449e4bdba93bdfb5cc89d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:09.853016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:09.853630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:09.855901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 955/1930] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:09 2022] DELETE /identity/v3/projects/f2969e7a601449e4bdba93bdfb5cc89d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:09.859707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:09.861160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:09.875284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407692d-7b9d-4361-a4b6-2b1f7a4a2a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:09.876056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407692d-7b9d-4361-a4b6-2b1f7a4a2a04 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:09.876407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407692d-7b9d-4361-a4b6-2b1f7a4a2a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:09.876763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407692d-7b9d-4361-a4b6-2b1f7a4a2a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:09.877462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407692d-7b9d-4361-a4b6-2b1f7a4a2a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:09.879705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407692d-7b9d-4361-a4b6-2b1f7a4a2a04 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:09.893296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 976/1931] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:09.894307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407692d-7b9d-4361-a4b6-2b1f7a4a2a04 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:09.896415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3407692d-7b9d-4361-a4b6-2b1f7a4a2a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:09.905256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:09.910095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407692d-7b9d-4361-a4b6-2b1f7a4a2a04 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:09.915557 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5ac08-7905-4f56-89c1-9d686c69abb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:09.916361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5ac08-7905-4f56-89c1-9d686c69abb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:09.916712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5ac08-7905-4f56-89c1-9d686c69abb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:09.917048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5ac08-7905-4f56-89c1-9d686c69abb1 admin admin] PATH_INFO: `/v3/projects/e9c0ab2759bc45d09289ab1e010a5afb/users/72a74b7f26e549f59b2023a375164626/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:09.933933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5ac08-7905-4f56-89c1-9d686c69abb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c0ab2759bc45d09289ab1e010a5afb, user_id=72a74b7f26e549f59b2023a375164626, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:09.936739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5ac08-7905-4f56-89c1-9d686c69abb1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:09.938208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a723e8-a1c3-4f31-9df6-0b3f48e85212 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:09.942624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:09.943464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:09.943819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:09.944165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] PATH_INFO: `/v3/users/81690cbbf33347798d72186e3075a117` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:09.945527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=81690cbbf33347798d72186e3075a117)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:09.947578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:10.002306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 977/1932] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:09 2022] PUT /identity/v3/projects/e9c0ab2759bc45d09289ab1e010a5afb/users/72a74b7f26e549f59b2023a375164626/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:10.006128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5ac08-7905-4f56-89c1-9d686c69abb1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:10.027740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:10.065155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:10.072291 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5ac08-7905-4f56-89c1-9d686c69abb1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:10.076947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b892b11-ad2d-4cef-9952-ecd7a485b7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:10.077552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b892b11-ad2d-4cef-9952-ecd7a485b7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:10.077834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b892b11-ad2d-4cef-9952-ecd7a485b7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:10.078133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b892b11-ad2d-4cef-9952-ecd7a485b7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:10.078807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b892b11-ad2d-4cef-9952-ecd7a485b7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:10.080381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b892b11-ad2d-4cef-9952-ecd7a485b7a8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:10.090906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 978/1933] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:10.095840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b892b11-ad2d-4cef-9952-ecd7a485b7a8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:10.105688 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b892b11-ad2d-4cef-9952-ecd7a485b7a8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:10.110451 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3daf7-d07b-42da-9ee3-c1ea445d2bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:10.111139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3daf7-d07b-42da-9ee3-c1ea445d2bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:10.111430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3daf7-d07b-42da-9ee3-c1ea445d2bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:10.111710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3daf7-d07b-42da-9ee3-c1ea445d2bb3 admin admin] PATH_INFO: `/v3/projects/e9c0ab2759bc45d09289ab1e010a5afb/users/72a74b7f26e549f59b2023a375164626/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:10.113762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3daf7-d07b-42da-9ee3-c1ea445d2bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c0ab2759bc45d09289ab1e010a5afb, user_id=72a74b7f26e549f59b2023a375164626, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:10.117096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3daf7-d07b-42da-9ee3-c1ea445d2bb3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:10.127494 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 979/1934] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:10 2022] PUT /identity/v3/projects/e9c0ab2759bc45d09289ab1e010a5afb/users/72a74b7f26e549f59b2023a375164626/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:10.138386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b5307-e7b9-4802-aabd-92b5ec90d4a1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:10.138703 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b5307-e7b9-4802-aabd-92b5ec90d4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:10.138993 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b5307-e7b9-4802-aabd-92b5ec90d4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:10.155966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81690cbbf33347798d72186e3075a117'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:10.171112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81690cbbf33347798d72186e3075a117'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:10.175848 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-117b5307-e7b9-4802-aabd-92b5ec90d4a1 None None] MFA Rules not processed for user `72a74b7f26e549f59b2023a375164626`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:10.177519 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-117b5307-e7b9-4802-aabd-92b5ec90d4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:10.179750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81690cbbf33347798d72186e3075a117'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:10.253896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 980/1935] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:10.259105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81690cbbf33347798d72186e3075a117'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:10.259841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81690cbbf33347798d72186e3075a117'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:10.266417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b5307-e7b9-4802-aabd-92b5ec90d4a1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:10.268151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-117b5307-e7b9-4802-aabd-92b5ec90d4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:10.269577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 956/1936] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:09 2022] DELETE /identity/v3/users/81690cbbf33347798d72186e3075a117 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:10.347753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b5307-e7b9-4802-aabd-92b5ec90d4a1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:10.362109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:10.400118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ee480-cfb3-4b1c-ba28-6c3e3788f155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:10.400831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ee480-cfb3-4b1c-ba28-6c3e3788f155 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:10.401112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ee480-cfb3-4b1c-ba28-6c3e3788f155 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:10.401380 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ee480-cfb3-4b1c-ba28-6c3e3788f155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:10.403223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e23ee480-cfb3-4b1c-ba28-6c3e3788f155 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:10.471618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ee480-cfb3-4b1c-ba28-6c3e3788f155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:10.473072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ee480-cfb3-4b1c-ba28-6c3e3788f155 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:10.484485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 981/1937] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:10.548482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:10.718118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:10.718118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:10.837593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27097ef1-5a3e-4061-85a6-21366a095fdb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:10.844551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:10.845815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:10.851091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:10.852112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] PATH_INFO: `/v3/projects/e8684766e3524f50875c0bb98ad491bb` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:10.855319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8684766e3524f50875c0bb98ad491bb)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:10.865302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:10.914515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:11.118001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8684766e3524f50875c0bb98ad491bb', 'request_id': 'req-dd80bf60-07a3-43c8-b957-657dfd04256b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:11.124569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8684766e3524f50875c0bb98ad491bb', 'request_id': 'req-dd80bf60-07a3-43c8-b957-657dfd04256b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:11.130999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8684766e3524f50875c0bb98ad491bb', 'request_id': 'req-dd80bf60-07a3-43c8-b957-657dfd04256b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:11.177602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ee480-cfb3-4b1c-ba28-6c3e3788f155 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:11.179134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e23ee480-cfb3-4b1c-ba28-6c3e3788f155 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:11.195359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ee480-cfb3-4b1c-ba28-6c3e3788f155 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:11.246823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] The token cache is being invalidate because project e8684766e3524f50875c0bb98ad491bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:11.250001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:11.250001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd80bf60-07a3-43c8-b957-657dfd04256b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:11.253744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 957/1938] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:10 2022] DELETE /identity/v3/projects/e8684766e3524f50875c0bb98ad491bb => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:11.258735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ca685-24d7-49dc-a545-cff1b5016928 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:11.260591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ca685-24d7-49dc-a545-cff1b5016928 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:11.261029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ca685-24d7-49dc-a545-cff1b5016928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:11.303727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9e4ca685-24d7-49dc-a545-cff1b5016928 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:11.305683 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9e4ca685-24d7-49dc-a545-cff1b5016928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:11.320711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b375d-a96c-4195-a7f2-e3059d0a326b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:11.321631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b375d-a96c-4195-a7f2-e3059d0a326b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:11.322057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b375d-a96c-4195-a7f2-e3059d0a326b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:11.322712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b375d-a96c-4195-a7f2-e3059d0a326b admin admin] PATH_INFO: `/v3/users/021d0fca0f4d4f17bed4d6c209af54ea` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:11.324151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b375d-a96c-4195-a7f2-e3059d0a326b admin admin] RBAC: Authorizing `identity:delete_user(user_id=021d0fca0f4d4f17bed4d6c209af54ea)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:11.326705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b375d-a96c-4195-a7f2-e3059d0a326b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:11.395484 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 958/1939] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:11.397298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9414baa-510f-4c56-bf37-2c34b94f3955 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:11.397584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9414baa-510f-4c56-bf37-2c34b94f3955 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:11.397864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9414baa-510f-4c56-bf37-2c34b94f3955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:11.420608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a9414baa-510f-4c56-bf37-2c34b94f3955 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:11.421842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a9414baa-510f-4c56-bf37-2c34b94f3955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:11.432475 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 959/1940] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:11.436944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9414baa-510f-4c56-bf37-2c34b94f3955 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:11.446073 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9414baa-510f-4c56-bf37-2c34b94f3955 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:11.455025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ed0e1-6fec-49a2-9d7e-0208c5b9ab48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:11.455025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ed0e1-6fec-49a2-9d7e-0208c5b9ab48 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:11.455025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ed0e1-6fec-49a2-9d7e-0208c5b9ab48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:11.455025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ed0e1-6fec-49a2-9d7e-0208c5b9ab48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:11.456202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ed0e1-6fec-49a2-9d7e-0208c5b9ab48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:11.456202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ed0e1-6fec-49a2-9d7e-0208c5b9ab48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:11.456202 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ed0e1-6fec-49a2-9d7e-0208c5b9ab48 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:11.473237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 960/1941] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:06:11.477914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ed0e1-6fec-49a2-9d7e-0208c5b9ab48 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:11.494554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ed0e1-6fec-49a2-9d7e-0208c5b9ab48 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:11.497047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ca178-e735-4db0-9c0b-9c45b92b08d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:11.497669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ca178-e735-4db0-9c0b-9c45b92b08d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:11.497946 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ca178-e735-4db0-9c0b-9c45b92b08d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:11.498215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ca178-e735-4db0-9c0b-9c45b92b08d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:11.498889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ca178-e735-4db0-9c0b-9c45b92b08d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:11.500950 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ca178-e735-4db0-9c0b-9c45b92b08d9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:11.583676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-552b375d-a96c-4195-a7f2-e3059d0a326b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '021d0fca0f4d4f17bed4d6c209af54ea'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:11.601205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 961/1942] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:11 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:11.608010 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ca178-e735-4db0-9c0b-9c45b92b08d9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:11.609838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-552b375d-a96c-4195-a7f2-e3059d0a326b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '021d0fca0f4d4f17bed4d6c209af54ea'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:11.612768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:11.622343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-552b375d-a96c-4195-a7f2-e3059d0a326b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '021d0fca0f4d4f17bed4d6c209af54ea'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:11.677764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:11.691521 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ca178-e735-4db0-9c0b-9c45b92b08d9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:11.693919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-552b375d-a96c-4195-a7f2-e3059d0a326b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '021d0fca0f4d4f17bed4d6c209af54ea'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:11.695244 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-552b375d-a96c-4195-a7f2-e3059d0a326b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '021d0fca0f4d4f17bed4d6c209af54ea'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:11.698538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded16aa2-1453-4b71-a54b-4b68e91f7282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:11.707085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded16aa2-1453-4b71-a54b-4b68e91f7282 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:11.707085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded16aa2-1453-4b71-a54b-4b68e91f7282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:11.707085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded16aa2-1453-4b71-a54b-4b68e91f7282 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:11.707085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded16aa2-1453-4b71-a54b-4b68e91f7282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:11.707085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded16aa2-1453-4b71-a54b-4b68e91f7282 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:11.707507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 982/1943] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:11 2022] DELETE /identity/v3/users/021d0fca0f4d4f17bed4d6c209af54ea => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:11.709782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef089990-c834-4557-9fa3-ed733f209801 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:11.710176 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef089990-c834-4557-9fa3-ed733f209801 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:11.711004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef089990-c834-4557-9fa3-ed733f209801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:11.751393 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ef089990-c834-4557-9fa3-ed733f209801 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:11.752812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ef089990-c834-4557-9fa3-ed733f209801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:11.819814 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 962/1944] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:11 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:11.823500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded16aa2-1453-4b71-a54b-4b68e91f7282 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:11.829631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ded16aa2-1453-4b71-a54b-4b68e91f7282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:11.860861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 983/1945] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:11.862103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded16aa2-1453-4b71-a54b-4b68e91f7282 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:11.874105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef089990-c834-4557-9fa3-ed733f209801 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:11.875764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47b4b1-889a-4f43-bc9e-38dbbd89ec87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:11.875928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ef089990-c834-4557-9fa3-ed733f209801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:11.876391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47b4b1-889a-4f43-bc9e-38dbbd89ec87 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:11.876681 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47b4b1-889a-4f43-bc9e-38dbbd89ec87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:11.876957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47b4b1-889a-4f43-bc9e-38dbbd89ec87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:11.884225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47b4b1-889a-4f43-bc9e-38dbbd89ec87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:11.884225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47b4b1-889a-4f43-bc9e-38dbbd89ec87 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:11.902892 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 963/1946] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:11.906705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47b4b1-889a-4f43-bc9e-38dbbd89ec87 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:11.930165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47b4b1-889a-4f43-bc9e-38dbbd89ec87 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:11.940948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0b515-f636-4670-b03d-e773c96eba84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:11.941570 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b515-f636-4670-b03d-e773c96eba84 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:11.941849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b515-f636-4670-b03d-e773c96eba84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:11.942709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b515-f636-4670-b03d-e773c96eba84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:11.943434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0b515-f636-4670-b03d-e773c96eba84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:11.945170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0b515-f636-4670-b03d-e773c96eba84 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:11.970496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 964/1947] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:11.987731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0b515-f636-4670-b03d-e773c96eba84 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:12.002204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef089990-c834-4557-9fa3-ed733f209801 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:12.009259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a115e7b-ed39-47ec-8406-446f949e1c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:12.010747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a115e7b-ed39-47ec-8406-446f949e1c2a service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:12.011313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a115e7b-ed39-47ec-8406-446f949e1c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:12.011513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a115e7b-ed39-47ec-8406-446f949e1c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:12.018758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0b515-f636-4670-b03d-e773c96eba84 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:12.028709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea0971-5c50-4722-a575-f51f6d8fe250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:12.029510 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea0971-5c50-4722-a575-f51f6d8fe250 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:12.029788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea0971-5c50-4722-a575-f51f6d8fe250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:12.030059 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea0971-5c50-4722-a575-f51f6d8fe250 admin admin] PATH_INFO: `/v3/projects/8b54e1e0edf74e53b77cce236388b9cb/users/2757da8a75de4210bbb945fa12335567/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:12.030938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a115e7b-ed39-47ec-8406-446f949e1c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:12.032529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a115e7b-ed39-47ec-8406-446f949e1c2a service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:12.046546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 984/1948] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:12.047766 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea0971-5c50-4722-a575-f51f6d8fe250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b54e1e0edf74e53b77cce236388b9cb, user_id=2757da8a75de4210bbb945fa12335567, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:12.050376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea0971-5c50-4722-a575-f51f6d8fe250 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:12.150279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 965/1949] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:11 2022] PUT /identity/v3/projects/8b54e1e0edf74e53b77cce236388b9cb/users/2757da8a75de4210bbb945fa12335567/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:12.156083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cc9b8-4d3a-434e-abcd-8378983e2339 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:12.156574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cc9b8-4d3a-434e-abcd-8378983e2339 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:12.157010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cc9b8-4d3a-434e-abcd-8378983e2339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:12.186479 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b77cc9b8-4d3a-434e-abcd-8378983e2339 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:12.187740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b77cc9b8-4d3a-434e-abcd-8378983e2339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:12.260422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 985/1950] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:12.270261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea0971-5c50-4722-a575-f51f6d8fe250 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:12.272232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9dea0971-5c50-4722-a575-f51f6d8fe250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:12.355267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea0971-5c50-4722-a575-f51f6d8fe250 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:12.359462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e91df-da16-4e0d-bc2a-8f79e99c281a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:12.360560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e91df-da16-4e0d-bc2a-8f79e99c281a service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:12.361549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e91df-da16-4e0d-bc2a-8f79e99c281a service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:12.361917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e91df-da16-4e0d-bc2a-8f79e99c281a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:12.376568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e91df-da16-4e0d-bc2a-8f79e99c281a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:12.380489 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e91df-da16-4e0d-bc2a-8f79e99c281a service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:12.396684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 966/1951] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:12.627381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cc9b8-4d3a-434e-abcd-8378983e2339 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:12.629329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b77cc9b8-4d3a-434e-abcd-8378983e2339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:12.641919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cc9b8-4d3a-434e-abcd-8378983e2339 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:12.647804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:12.648710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:12.649201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:12.649628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] PATH_INFO: `/v3/projects/964cf92762cd4c8fa1d4ccabcae89fc2` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:12.651099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=964cf92762cd4c8fa1d4ccabcae89fc2)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:12.657352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:12.702240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:12.822360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '964cf92762cd4c8fa1d4ccabcae89fc2', 'request_id': 'req-0e609995-0ae0-441d-90bc-ba3c5786ce4e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:12.830018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '964cf92762cd4c8fa1d4ccabcae89fc2', 'request_id': 'req-0e609995-0ae0-441d-90bc-ba3c5786ce4e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:12.841586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '964cf92762cd4c8fa1d4ccabcae89fc2', 'request_id': 'req-0e609995-0ae0-441d-90bc-ba3c5786ce4e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:12.988322 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] The token cache is being invalidate because project 964cf92762cd4c8fa1d4ccabcae89fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:12.988623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:12.989260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0e609995-0ae0-441d-90bc-ba3c5786ce4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:12.991134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 986/1952] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:12 2022] DELETE /identity/v3/projects/964cf92762cd4c8fa1d4ccabcae89fc2 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:12.994957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e91df-da16-4e0d-bc2a-8f79e99c281a service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:13.005727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e91df-da16-4e0d-bc2a-8f79e99c281a service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:13.071114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:13.071896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:13.072272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:13.072646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] PATH_INFO: `/v3/users/72a74b7f26e549f59b2023a375164626` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:13.073919 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a74b7f26e549f59b2023a375164626)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:13.076007 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:13.294895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:13.301179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a74b7f26e549f59b2023a375164626'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:13.313624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a74b7f26e549f59b2023a375164626'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:13.321289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a74b7f26e549f59b2023a375164626'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:13.397984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a74b7f26e549f59b2023a375164626'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:13.398760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a74b7f26e549f59b2023a375164626'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:13.414555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 967/1953] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:12 2022] DELETE /identity/v3/users/72a74b7f26e549f59b2023a375164626 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:13.917141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593e00b-94e2-4c6c-816e-a068dd8b2f17 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:13.919014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593e00b-94e2-4c6c-816e-a068dd8b2f17 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:13.919519 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593e00b-94e2-4c6c-816e-a068dd8b2f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:13.933372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:13.934988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:13.998062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7593e00b-94e2-4c6c-816e-a068dd8b2f17 None None] MFA Rules not processed for user `2757da8a75de4210bbb945fa12335567`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:14.000984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7593e00b-94e2-4c6c-816e-a068dd8b2f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:14.095605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd6d39-805c-42cb-8f0f-7fa873881c1c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.100046 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.100896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.101174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.101700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] PATH_INFO: `/v3/projects/e9c0ab2759bc45d09289ab1e010a5afb` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.102757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c0ab2759bc45d09289ab1e010a5afb)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.104955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.132871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 987/1954] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:14.140367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593e00b-94e2-4c6c-816e-a068dd8b2f17 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.141703 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7593e00b-94e2-4c6c-816e-a068dd8b2f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:14.155443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593e00b-94e2-4c6c-816e-a068dd8b2f17 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.160186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e6f39-e939-4395-a074-6925736aa6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.161451 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e6f39-e939-4395-a074-6925736aa6bc admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.161728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e6f39-e939-4395-a074-6925736aa6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.161995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e6f39-e939-4395-a074-6925736aa6bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.162620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e6f39-e939-4395-a074-6925736aa6bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.164594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e6f39-e939-4395-a074-6925736aa6bc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.228388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c0ab2759bc45d09289ab1e010a5afb', 'request_id': 'req-44d14981-2f03-4feb-985a-3e639fed939f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:14.234621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c0ab2759bc45d09289ab1e010a5afb', 'request_id': 'req-44d14981-2f03-4feb-985a-3e639fed939f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:14.240204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c0ab2759bc45d09289ab1e010a5afb', 'request_id': 'req-44d14981-2f03-4feb-985a-3e639fed939f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:14.276079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 988/1955] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:14 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:14.280054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e6f39-e939-4395-a074-6925736aa6bc admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.316210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] The token cache is being invalidate because project e9c0ab2759bc45d09289ab1e010a5afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:14.316405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:14.317532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-44d14981-2f03-4feb-985a-3e639fed939f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:14.319218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 968/1956] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:13 2022] DELETE /identity/v3/projects/e9c0ab2759bc45d09289ab1e010a5afb => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:14.323850 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326054-cdc3-44de-9dad-bc797c7519b7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.324124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326054-cdc3-44de-9dad-bc797c7519b7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.324395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326054-cdc3-44de-9dad-bc797c7519b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.344044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e6f39-e939-4395-a074-6925736aa6bc admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.344228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c6326054-cdc3-44de-9dad-bc797c7519b7 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:14.345322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c6326054-cdc3-44de-9dad-bc797c7519b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:14.348825 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d67563-0c02-44d6-8ec2-6715aec725b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.352843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d67563-0c02-44d6-8ec2-6715aec725b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.352843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d67563-0c02-44d6-8ec2-6715aec725b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.352843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d67563-0c02-44d6-8ec2-6715aec725b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.352843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d67563-0c02-44d6-8ec2-6715aec725b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.352843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d67563-0c02-44d6-8ec2-6715aec725b1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.353266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 969/1957] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:14.361196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d030c5-f632-4e58-a8d4-45f726de9e21 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.361483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d030c5-f632-4e58-a8d4-45f726de9e21 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.361765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d030c5-f632-4e58-a8d4-45f726de9e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.384891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-36d030c5-f632-4e58-a8d4-45f726de9e21 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:14.386101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-36d030c5-f632-4e58-a8d4-45f726de9e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:14.394508 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 970/1958] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:14.403558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d030c5-f632-4e58-a8d4-45f726de9e21 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.412959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d030c5-f632-4e58-a8d4-45f726de9e21 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.417535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1949c54-ebd1-455c-ad94-fc266dc2b3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.418157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1949c54-ebd1-455c-ad94-fc266dc2b3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.418439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1949c54-ebd1-455c-ad94-fc266dc2b3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.418739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1949c54-ebd1-455c-ad94-fc266dc2b3f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.419376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1949c54-ebd1-455c-ad94-fc266dc2b3f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:14.419831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1949c54-ebd1-455c-ad94-fc266dc2b3f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.421454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1949c54-ebd1-455c-ad94-fc266dc2b3f7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.433203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 971/1959] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:06:14.437465 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1949c54-ebd1-455c-ad94-fc266dc2b3f7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.447532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1949c54-ebd1-455c-ad94-fc266dc2b3f7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.448697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 989/1960] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:14 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:14.452813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d67563-0c02-44d6-8ec2-6715aec725b1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.454589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-08d67563-0c02-44d6-8ec2-6715aec725b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:14.455090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ad9cf-8e36-4cb8-ae38-b99231b63e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.455686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ad9cf-8e36-4cb8-ae38-b99231b63e81 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.455963 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ad9cf-8e36-4cb8-ae38-b99231b63e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.456232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ad9cf-8e36-4cb8-ae38-b99231b63e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.456857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ad9cf-8e36-4cb8-ae38-b99231b63e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.459431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ad9cf-8e36-4cb8-ae38-b99231b63e81 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.468249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d67563-0c02-44d6-8ec2-6715aec725b1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.473246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98168492-035e-4734-945c-809ace9383b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.473875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98168492-035e-4734-945c-809ace9383b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.474158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98168492-035e-4734-945c-809ace9383b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.474424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98168492-035e-4734-945c-809ace9383b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.475285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98168492-035e-4734-945c-809ace9383b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.476959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98168492-035e-4734-945c-809ace9383b6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.479344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:14.487972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 990/1961] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:14.492041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98168492-035e-4734-945c-809ace9383b6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.501523 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98168492-035e-4734-945c-809ace9383b6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.506647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad918e0-7b7e-44df-9d4d-187817a6d2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.508487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad918e0-7b7e-44df-9d4d-187817a6d2d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.508487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad918e0-7b7e-44df-9d4d-187817a6d2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.508487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad918e0-7b7e-44df-9d4d-187817a6d2d1 admin admin] PATH_INFO: `/v3/projects/13c8229ddcc04e29825df27ad96ad7f1/users/3d9881a127cb45fca47494914985a44f/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.524175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad918e0-7b7e-44df-9d4d-187817a6d2d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c8229ddcc04e29825df27ad96ad7f1, user_id=3d9881a127cb45fca47494914985a44f, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.526947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad918e0-7b7e-44df-9d4d-187817a6d2d1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.591095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:14.598447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 972/1962] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:14 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:14.602389 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ad9cf-8e36-4cb8-ae38-b99231b63e81 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.631905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 991/1963] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:14 2022] PUT /identity/v3/projects/13c8229ddcc04e29825df27ad96ad7f1/users/3d9881a127cb45fca47494914985a44f/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:14.636764 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad918e0-7b7e-44df-9d4d-187817a6d2d1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.683479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ad9cf-8e36-4cb8-ae38-b99231b63e81 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.725042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad918e0-7b7e-44df-9d4d-187817a6d2d1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.732307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f9871-1f0d-45f6-ae50-ca6314d3ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.732930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f9871-1f0d-45f6-ae50-ca6314d3ec45 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.733548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f9871-1f0d-45f6-ae50-ca6314d3ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.733548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f9871-1f0d-45f6-ae50-ca6314d3ec45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.734169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f9871-1f0d-45f6-ae50-ca6314d3ec45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.741683 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f9871-1f0d-45f6-ae50-ca6314d3ec45 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.746793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2ad35-e3c1-44b7-b745-026ecc81c750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.747047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2ad35-e3c1-44b7-b745-026ecc81c750 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.747335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2ad35-e3c1-44b7-b745-026ecc81c750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.747613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2ad35-e3c1-44b7-b745-026ecc81c750 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.748334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2ad35-e3c1-44b7-b745-026ecc81c750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.750443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2ad35-e3c1-44b7-b745-026ecc81c750 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.753299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 992/1964] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:14.758121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f9871-1f0d-45f6-ae50-ca6314d3ec45 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.770232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f9871-1f0d-45f6-ae50-ca6314d3ec45 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.774961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16285f-23c8-4971-80ac-949ffa6018bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.775747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16285f-23c8-4971-80ac-949ffa6018bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.776030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16285f-23c8-4971-80ac-949ffa6018bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.776310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16285f-23c8-4971-80ac-949ffa6018bf admin admin] PATH_INFO: `/v3/projects/13c8229ddcc04e29825df27ad96ad7f1/users/3d9881a127cb45fca47494914985a44f/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.778246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16285f-23c8-4971-80ac-949ffa6018bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c8229ddcc04e29825df27ad96ad7f1, user_id=3d9881a127cb45fca47494914985a44f, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.781192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16285f-23c8-4971-80ac-949ffa6018bf admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.794391 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 993/1965] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:14 2022] PUT /identity/v3/projects/13c8229ddcc04e29825df27ad96ad7f1/users/3d9881a127cb45fca47494914985a44f/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:14.842266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 973/1966] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:14 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:14.853121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16285f-23c8-4971-80ac-949ffa6018bf admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.922586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16285f-23c8-4971-80ac-949ffa6018bf admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.928159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49aaaa-daf4-494a-b068-41f00f4a6fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.928888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49aaaa-daf4-494a-b068-41f00f4a6fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.929243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49aaaa-daf4-494a-b068-41f00f4a6fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.929523 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49aaaa-daf4-494a-b068-41f00f4a6fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.930716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49aaaa-daf4-494a-b068-41f00f4a6fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.932035 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49aaaa-daf4-494a-b068-41f00f4a6fc6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.945775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 994/1967] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:14.946976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2ad35-e3c1-44b7-b745-026ecc81c750 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.957175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2ad35-e3c1-44b7-b745-026ecc81c750 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:14.963479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3ff0e-0587-4b41-9e03-09b3252baebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:14.963997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3ff0e-0587-4b41-9e03-09b3252baebe admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:14.964603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3ff0e-0587-4b41-9e03-09b3252baebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:14.964883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3ff0e-0587-4b41-9e03-09b3252baebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:14.965536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3ff0e-0587-4b41-9e03-09b3252baebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:14.967240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3ff0e-0587-4b41-9e03-09b3252baebe admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:14.978658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 974/1968] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:14.983008 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49aaaa-daf4-494a-b068-41f00f4a6fc6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:14.997271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49aaaa-daf4-494a-b068-41f00f4a6fc6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:15.003558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09871c4c-d584-4db2-9042-8f4b18756a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:15.003720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09871c4c-d584-4db2-9042-8f4b18756a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:15.004009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09871c4c-d584-4db2-9042-8f4b18756a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:15.004284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09871c4c-d584-4db2-9042-8f4b18756a00 admin admin] PATH_INFO: `/v3/projects/bb39ded310e3409184bc4a8adb1a9a40/users/28e0983c384c47ca9ac2f04149956753/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:15.023945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09871c4c-d584-4db2-9042-8f4b18756a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb39ded310e3409184bc4a8adb1a9a40, user_id=28e0983c384c47ca9ac2f04149956753, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:15.027269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09871c4c-d584-4db2-9042-8f4b18756a00 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:15.122854 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 995/1969] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:14 2022] PUT /identity/v3/projects/bb39ded310e3409184bc4a8adb1a9a40/users/28e0983c384c47ca9ac2f04149956753/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:15.128655 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94c5e-097b-4b19-af36-88eb48cb3300 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:15.129197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94c5e-097b-4b19-af36-88eb48cb3300 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:15.129897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94c5e-097b-4b19-af36-88eb48cb3300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:15.173884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fda94c5e-097b-4b19-af36-88eb48cb3300 None None] MFA Rules not processed for user `28e0983c384c47ca9ac2f04149956753`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:15.175478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fda94c5e-097b-4b19-af36-88eb48cb3300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:15.292609 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09871c4c-d584-4db2-9042-8f4b18756a00 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:15.295177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 975/1970] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:15.366263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09871c4c-d584-4db2-9042-8f4b18756a00 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:15.371637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6496d-702a-46b7-8faf-5544ab9094b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:15.372083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6496d-702a-46b7-8faf-5544ab9094b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:15.372367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6496d-702a-46b7-8faf-5544ab9094b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:15.372632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6496d-702a-46b7-8faf-5544ab9094b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:15.373275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6496d-702a-46b7-8faf-5544ab9094b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:15.379649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6496d-702a-46b7-8faf-5544ab9094b4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:15.477054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:15.564152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 996/1971] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:15 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:15.565023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda94c5e-097b-4b19-af36-88eb48cb3300 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:15.565023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda94c5e-097b-4b19-af36-88eb48cb3300 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:15.574627 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:15.644997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa7ccb-9f8d-4804-93b4-a310f6a2e5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:15.645757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa7ccb-9f8d-4804-93b4-a310f6a2e5b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:15.646114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa7ccb-9f8d-4804-93b4-a310f6a2e5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:15.646454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa7ccb-9f8d-4804-93b4-a310f6a2e5b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:15.647209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa7ccb-9f8d-4804-93b4-a310f6a2e5b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:15.649206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa7ccb-9f8d-4804-93b4-a310f6a2e5b7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:15.770538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 976/1972] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:15 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:15.775196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6496d-702a-46b7-8faf-5544ab9094b4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:15.786837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6496d-702a-46b7-8faf-5544ab9094b4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:15.801148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087c8e2-22cf-4ce1-88f3-5cf1e115ade7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:15.802883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087c8e2-22cf-4ce1-88f3-5cf1e115ade7 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:15.803436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087c8e2-22cf-4ce1-88f3-5cf1e115ade7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:15.803951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087c8e2-22cf-4ce1-88f3-5cf1e115ade7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:15.804823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087c8e2-22cf-4ce1-88f3-5cf1e115ade7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:15.808606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087c8e2-22cf-4ce1-88f3-5cf1e115ade7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:15.822989 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 997/1973] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:15.827251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa7ccb-9f8d-4804-93b4-a310f6a2e5b7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:15.841714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa7ccb-9f8d-4804-93b4-a310f6a2e5b7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:15.846891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47da9aa-36a5-49b3-af24-80daf4a1fcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:15.847632 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47da9aa-36a5-49b3-af24-80daf4a1fcd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:15.848962 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47da9aa-36a5-49b3-af24-80daf4a1fcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:15.849179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47da9aa-36a5-49b3-af24-80daf4a1fcd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:15.850097 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47da9aa-36a5-49b3-af24-80daf4a1fcd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:15.851793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47da9aa-36a5-49b3-af24-80daf4a1fcd0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:15.866019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 977/1974] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:15.870382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087c8e2-22cf-4ce1-88f3-5cf1e115ade7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:15.881699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47da9aa-36a5-49b3-af24-80daf4a1fcd0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:15.884869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a47da9aa-36a5-49b3-af24-80daf4a1fcd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:15.887654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087c8e2-22cf-4ce1-88f3-5cf1e115ade7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:15.892520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cd0af-f2ca-4870-ab9b-cb314d692ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:15.893214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cd0af-f2ca-4870-ab9b-cb314d692ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:15.893516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cd0af-f2ca-4870-ab9b-cb314d692ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:15.893816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cd0af-f2ca-4870-ab9b-cb314d692ff2 admin admin] PATH_INFO: `/v3/projects/c2f4c815d18148319447c08261151ce9/users/e7b4fada64d047bb83dd9a771f20b3ec/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:15.901336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47da9aa-36a5-49b3-af24-80daf4a1fcd0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:15.907642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:15.908730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:15.909235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:15.909922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] PATH_INFO: `/v3/users/5e123f7323f3463fa812e51edf28d2ee` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:15.910279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cd0af-f2ca-4870-ab9b-cb314d692ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f4c815d18148319447c08261151ce9, user_id=e7b4fada64d047bb83dd9a771f20b3ec, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:15.912052 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e123f7323f3463fa812e51edf28d2ee)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:15.913956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cd0af-f2ca-4870-ab9b-cb314d692ff2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:15.914650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:16.037082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 998/1975] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:15 2022] PUT /identity/v3/projects/c2f4c815d18148319447c08261151ce9/users/e7b4fada64d047bb83dd9a771f20b3ec/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:16.048919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6624b-7c0a-4590-88b6-85531c1197d3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:16.048919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6624b-7c0a-4590-88b6-85531c1197d3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:16.048919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6624b-7c0a-4590-88b6-85531c1197d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:16.076948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b8e6624b-7c0a-4590-88b6-85531c1197d3 None None] MFA Rules not processed for user `e7b4fada64d047bb83dd9a771f20b3ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:16.078293 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b8e6624b-7c0a-4590-88b6-85531c1197d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:16.177041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 999/1976] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:16.181504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6624b-7c0a-4590-88b6-85531c1197d3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:16.242197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e123f7323f3463fa812e51edf28d2ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:16.313753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6624b-7c0a-4590-88b6-85531c1197d3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:16.323904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568589ec-7ec1-41d0-9c90-3d7e0e2c3a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:16.324266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568589ec-7ec1-41d0-9c90-3d7e0e2c3a75 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:16.324571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568589ec-7ec1-41d0-9c90-3d7e0e2c3a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:16.324875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568589ec-7ec1-41d0-9c90-3d7e0e2c3a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:16.325572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568589ec-7ec1-41d0-9c90-3d7e0e2c3a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:16.327913 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568589ec-7ec1-41d0-9c90-3d7e0e2c3a75 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:16.343169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e123f7323f3463fa812e51edf28d2ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:16.353668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e123f7323f3463fa812e51edf28d2ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:17.419760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e123f7323f3463fa812e51edf28d2ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:17.421220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e123f7323f3463fa812e51edf28d2ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:17.428850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1000/1977] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:16 2022] POST /identity/v3/projects => generated 383 bytes in 1248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:17.430846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b053e27-b1de-427a-9c2b-b7f244880dc1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:17.431286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b053e27-b1de-427a-9c2b-b7f244880dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:17.431998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b053e27-b1de-427a-9c2b-b7f244880dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:17.435199 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 978/1978] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:15 2022] DELETE /identity/v3/users/5e123f7323f3463fa812e51edf28d2ee => generated 0 bytes in 1552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:17.436530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:17.439065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:17.469592 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8b053e27-b1de-427a-9c2b-b7f244880dc1 None None] MFA Rules not processed for user `3d9881a127cb45fca47494914985a44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:17.470857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8b053e27-b1de-427a-9c2b-b7f244880dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:17.524063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff61f4f-533c-4402-9c8f-62598ec56882 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:17.542168 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7287d-30c1-4ad3-a8d0-6018ec86155f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:17.543078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7287d-30c1-4ad3-a8d0-6018ec86155f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:17.543533 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:17.547793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7287d-30c1-4ad3-a8d0-6018ec86155f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:17.549136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7287d-30c1-4ad3-a8d0-6018ec86155f admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:17.550328 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7287d-30c1-4ad3-a8d0-6018ec86155f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:17.556799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7287d-30c1-4ad3-a8d0-6018ec86155f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:17.604831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1001/1979] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:17.606358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e638a9e-4cbb-4a12-aeb9-82a481e44f22 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:17.606986 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e638a9e-4cbb-4a12-aeb9-82a481e44f22 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:17.607454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e638a9e-4cbb-4a12-aeb9-82a481e44f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:17.633830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2e638a9e-4cbb-4a12-aeb9-82a481e44f22 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:17.635603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2e638a9e-4cbb-4a12-aeb9-82a481e44f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:17.645864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1002/1980] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:17.647156 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e638a9e-4cbb-4a12-aeb9-82a481e44f22 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:17.649663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2e638a9e-4cbb-4a12-aeb9-82a481e44f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:17.701249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 979/1981] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:17 2022] POST /identity/v3/users => generated 298 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:17.702516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7287d-30c1-4ad3-a8d0-6018ec86155f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:17.704641 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c5c7287d-30c1-4ad3-a8d0-6018ec86155f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:17.726502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e638a9e-4cbb-4a12-aeb9-82a481e44f22 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:17.740520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970d08d-6274-41e3-843b-ffeadc14fcfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:17.741153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970d08d-6274-41e3-843b-ffeadc14fcfb service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:17.741433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970d08d-6274-41e3-843b-ffeadc14fcfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:17.741702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970d08d-6274-41e3-843b-ffeadc14fcfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:17.744991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d970d08d-6274-41e3-843b-ffeadc14fcfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:17.802532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7287d-30c1-4ad3-a8d0-6018ec86155f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:17.807262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c2342-68fa-46e5-9968-b68e24325075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:17.807844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c2342-68fa-46e5-9968-b68e24325075 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:17.808256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c2342-68fa-46e5-9968-b68e24325075 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:17.808536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c2342-68fa-46e5-9968-b68e24325075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:17.815251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970d08d-6274-41e3-843b-ffeadc14fcfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:17.816676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970d08d-6274-41e3-843b-ffeadc14fcfb service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:17.818334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c2342-68fa-46e5-9968-b68e24325075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:17.819885 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c2342-68fa-46e5-9968-b68e24325075 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:17.826161 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1003/1982] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:17.827043 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970d08d-6274-41e3-843b-ffeadc14fcfb service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:17.853772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 980/1983] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:17.853868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970d08d-6274-41e3-843b-ffeadc14fcfb service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:17.853868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b921412-655a-4f62-8e34-53b9cd5b2513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:17.853868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b921412-655a-4f62-8e34-53b9cd5b2513 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:17.853868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b921412-655a-4f62-8e34-53b9cd5b2513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:17.854175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b921412-655a-4f62-8e34-53b9cd5b2513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:17.854175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b921412-655a-4f62-8e34-53b9cd5b2513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:17.854175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b921412-655a-4f62-8e34-53b9cd5b2513 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:17.862930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1004/1984] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:17.865451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:17.867466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c2342-68fa-46e5-9968-b68e24325075 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:17.879505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c2342-68fa-46e5-9968-b68e24325075 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:17.885611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254422fe-b36d-49d0-b3cc-37061213eb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:17.886293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254422fe-b36d-49d0-b3cc-37061213eb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:17.886571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254422fe-b36d-49d0-b3cc-37061213eb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:17.886877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254422fe-b36d-49d0-b3cc-37061213eb9c admin admin] PATH_INFO: `/v3/projects/3bd5bd1ddfef47919f0f3e37b790b746/users/eb815a3d46bb447985cd021def5dccb1/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:17.907871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254422fe-b36d-49d0-b3cc-37061213eb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd5bd1ddfef47919f0f3e37b790b746, user_id=eb815a3d46bb447985cd021def5dccb1, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:17.912270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254422fe-b36d-49d0-b3cc-37061213eb9c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:18.017653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 981/1985] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:17 2022] PUT /identity/v3/projects/3bd5bd1ddfef47919f0f3e37b790b746/users/eb815a3d46bb447985cd021def5dccb1/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:18.021698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b921412-655a-4f62-8e34-53b9cd5b2513 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:18.032323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b921412-655a-4f62-8e34-53b9cd5b2513 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:18.116931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18082adc-d632-4032-a892-433df59bb916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:18.118251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18082adc-d632-4032-a892-433df59bb916 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:18.118613 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18082adc-d632-4032-a892-433df59bb916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:18.119072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18082adc-d632-4032-a892-433df59bb916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:18.120010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18082adc-d632-4032-a892-433df59bb916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:18.121763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18082adc-d632-4032-a892-433df59bb916 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:18.139426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1005/1986] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:18.143325 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254422fe-b36d-49d0-b3cc-37061213eb9c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:18.155731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254422fe-b36d-49d0-b3cc-37061213eb9c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:18.163483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a8495-5355-4679-9f3d-80c706d7d247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:18.164173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a8495-5355-4679-9f3d-80c706d7d247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:18.164464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a8495-5355-4679-9f3d-80c706d7d247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:18.164880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a8495-5355-4679-9f3d-80c706d7d247 admin admin] PATH_INFO: `/v3/projects/3bd5bd1ddfef47919f0f3e37b790b746/users/eb815a3d46bb447985cd021def5dccb1/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:18.166764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a8495-5355-4679-9f3d-80c706d7d247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd5bd1ddfef47919f0f3e37b790b746, user_id=eb815a3d46bb447985cd021def5dccb1, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:18.173959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a8495-5355-4679-9f3d-80c706d7d247 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:18.199515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 982/1987] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:18 2022] PUT /identity/v3/projects/3bd5bd1ddfef47919f0f3e37b790b746/users/eb815a3d46bb447985cd021def5dccb1/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:18.205776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219eaf66-df6b-4251-9802-09c34c8f6a3d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:18.206023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219eaf66-df6b-4251-9802-09c34c8f6a3d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:18.206315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219eaf66-df6b-4251-9802-09c34c8f6a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:18.272976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-219eaf66-df6b-4251-9802-09c34c8f6a3d None None] MFA Rules not processed for user `eb815a3d46bb447985cd021def5dccb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:18.274297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-219eaf66-df6b-4251-9802-09c34c8f6a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:18.340960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a8495-5355-4679-9f3d-80c706d7d247 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:18.343383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-803a8495-5355-4679-9f3d-80c706d7d247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:18.373154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1006/1988] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:18.442349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219eaf66-df6b-4251-9802-09c34c8f6a3d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:18.442349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-219eaf66-df6b-4251-9802-09c34c8f6a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:18.466427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a8495-5355-4679-9f3d-80c706d7d247 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:18.471671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:18.472375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:18.472668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:18.472948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] PATH_INFO: `/v3/projects/5e900d236439454284220179da162c08` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:18.474718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e900d236439454284220179da162c08)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:18.476978 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:18.499935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219eaf66-df6b-4251-9802-09c34c8f6a3d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:18.503545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf805a7-088f-4c13-a94d-4fb570da10c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:18.504167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf805a7-088f-4c13-a94d-4fb570da10c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:18.504512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf805a7-088f-4c13-a94d-4fb570da10c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:18.504789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf805a7-088f-4c13-a94d-4fb570da10c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:18.506714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bbf805a7-088f-4c13-a94d-4fb570da10c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:18.567590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf805a7-088f-4c13-a94d-4fb570da10c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:18.569172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf805a7-088f-4c13-a94d-4fb570da10c8 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:18.578048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1007/1989] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:18.615236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e900d236439454284220179da162c08', 'request_id': 'req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:18.619973 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e900d236439454284220179da162c08', 'request_id': 'req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:18.624599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e900d236439454284220179da162c08', 'request_id': 'req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:18.751847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] The token cache is being invalidate because project 5e900d236439454284220179da162c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:18.752438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:18.753392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:18.756139 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 983/1990] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:18 2022] DELETE /identity/v3/projects/5e900d236439454284220179da162c08 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:18.760167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf805a7-088f-4c13-a94d-4fb570da10c8 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:18.773510 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf805a7-088f-4c13-a94d-4fb570da10c8 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:18.835434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:18.836036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:18.836295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:18.836806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] PATH_INFO: `/v3/users/d1cef61d4f2f4bc1b72af0cf591f8231` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:18.839942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1cef61d4f2f4bc1b72af0cf591f8231)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:18.841907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:19.106279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1cef61d4f2f4bc1b72af0cf591f8231'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:19.124032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1cef61d4f2f4bc1b72af0cf591f8231'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:19.126931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1cef61d4f2f4bc1b72af0cf591f8231'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:19.199384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1cef61d4f2f4bc1b72af0cf591f8231'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:19.200796 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:19.201453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1cef61d4f2f4bc1b72af0cf591f8231'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:19.208037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1008/1991] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:18 2022] DELETE /identity/v3/users/d1cef61d4f2f4bc1b72af0cf591f8231 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:19.677927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:19.679558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:19.694106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951cb81-32ab-45b8-8c2c-7ef0c19d022c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:19.761927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:19.762667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:19.762981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:19.763256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] PATH_INFO: `/v3/projects/1885aea0d2324b9cbee2b441339b0a15` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:19.764422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1885aea0d2324b9cbee2b441339b0a15)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:19.766348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:19.870293 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:19.871830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:19.896727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1885aea0d2324b9cbee2b441339b0a15', 'request_id': 'req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:19.905004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1885aea0d2324b9cbee2b441339b0a15', 'request_id': 'req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:19.912066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1885aea0d2324b9cbee2b441339b0a15', 'request_id': 'req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:19.936054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b5437-6423-4e39-81ea-775c9f007ae0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:19.984983 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f05802-7593-48cf-807e-9028c701d284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:19.985922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f05802-7593-48cf-807e-9028c701d284 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:19.986202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f05802-7593-48cf-807e-9028c701d284 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:19.986479 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f05802-7593-48cf-807e-9028c701d284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:19.988406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-56f05802-7593-48cf-807e-9028c701d284 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:19.990092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] The token cache is being invalidate because project 1885aea0d2324b9cbee2b441339b0a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:19.990395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:19.991216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81cd1a5-d71f-49f7-8b27-9a89db68aacc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:19.996259 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 984/1992] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:19 2022] DELETE /identity/v3/projects/1885aea0d2324b9cbee2b441339b0a15 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:20.005307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df7cb7-b0f6-4e3f-abe0-e8edcee751ab None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.005307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df7cb7-b0f6-4e3f-abe0-e8edcee751ab None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.005307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df7cb7-b0f6-4e3f-abe0-e8edcee751ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.026402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-05df7cb7-b0f6-4e3f-abe0-e8edcee751ab None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:20.028764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-05df7cb7-b0f6-4e3f-abe0-e8edcee751ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:20.061474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f05802-7593-48cf-807e-9028c701d284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:20.063147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f05802-7593-48cf-807e-9028c701d284 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:20.073994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1009/1993] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:20.098845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 985/1994] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:20.107923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c72db8-f816-4223-9e98-c189581f7a4e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.108135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c72db8-f816-4223-9e98-c189581f7a4e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.108501 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c72db8-f816-4223-9e98-c189581f7a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.131154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-46c72db8-f816-4223-9e98-c189581f7a4e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:20.132401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-46c72db8-f816-4223-9e98-c189581f7a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:20.140574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1010/1995] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:20.145727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df7cb7-b0f6-4e3f-abe0-e8edcee751ab None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:20.156042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df7cb7-b0f6-4e3f-abe0-e8edcee751ab None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:20.164615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb516e-ded2-4ec1-8cf1-23d013e3eb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:20.164615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb516e-ded2-4ec1-8cf1-23d013e3eb50 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.164615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb516e-ded2-4ec1-8cf1-23d013e3eb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.164615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb516e-ded2-4ec1-8cf1-23d013e3eb50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.164974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb516e-ded2-4ec1-8cf1-23d013e3eb50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:20.164974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb516e-ded2-4ec1-8cf1-23d013e3eb50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:20.164974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb516e-ded2-4ec1-8cf1-23d013e3eb50 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:20.176680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 986/1996] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:06:20.181294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c72db8-f816-4223-9e98-c189581f7a4e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:20.190517 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c72db8-f816-4223-9e98-c189581f7a4e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:20.195751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15362aa-f090-40c9-8f39-5c382dcc102e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:20.196349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15362aa-f090-40c9-8f39-5c382dcc102e admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.196627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15362aa-f090-40c9-8f39-5c382dcc102e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.196897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15362aa-f090-40c9-8f39-5c382dcc102e admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.197526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15362aa-f090-40c9-8f39-5c382dcc102e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:20.199436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15362aa-f090-40c9-8f39-5c382dcc102e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:20.332581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1011/1997] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:20 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:20.336874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb516e-ded2-4ec1-8cf1-23d013e3eb50 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:20.345160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=3,keystone:SELECT=91,keystone:DELETE=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:20.349624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb516e-ded2-4ec1-8cf1-23d013e3eb50 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:20.406262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4593f-52be-4258-beef-212836b4ecfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:20.406954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4593f-52be-4258-beef-212836b4ecfa admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.407214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4593f-52be-4258-beef-212836b4ecfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.407492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4593f-52be-4258-beef-212836b4ecfa admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.408138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4593f-52be-4258-beef-212836b4ecfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:20.424575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4593f-52be-4258-beef-212836b4ecfa admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:20.501915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 987/1998] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:20 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:20.506953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15362aa-f090-40c9-8f39-5c382dcc102e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:20.518156 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15362aa-f090-40c9-8f39-5c382dcc102e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:20.523803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4593f-52be-4258-beef-212836b4ecfa admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:20.526760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-04c4593f-52be-4258-beef-212836b4ecfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:20.533721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679677c2-75e4-4047-9346-82149b35f314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:20.535477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679677c2-75e4-4047-9346-82149b35f314 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.535969 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679677c2-75e4-4047-9346-82149b35f314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.536255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679677c2-75e4-4047-9346-82149b35f314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.536957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679677c2-75e4-4047-9346-82149b35f314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:20.538945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679677c2-75e4-4047-9346-82149b35f314 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:20.554288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1012/1999] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:20.555426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679677c2-75e4-4047-9346-82149b35f314 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:20.559098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4593f-52be-4258-beef-212836b4ecfa admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:20.563611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe63442-2a03-4e24-9be0-cf220bc0a15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:20.564490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe63442-2a03-4e24-9be0-cf220bc0a15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.565513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe63442-2a03-4e24-9be0-cf220bc0a15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.565888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe63442-2a03-4e24-9be0-cf220bc0a15c admin admin] PATH_INFO: `/v3/users/2e4fdaf1205c4816b5c477edc2097ddb` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.566439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679677c2-75e4-4047-9346-82149b35f314 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:20.567943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe63442-2a03-4e24-9be0-cf220bc0a15c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e4fdaf1205c4816b5c477edc2097ddb)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:20.570278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe63442-2a03-4e24-9be0-cf220bc0a15c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:20.572242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577a86-8613-4c2e-a6b7-4724bcecea01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:20.572976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577a86-8613-4c2e-a6b7-4724bcecea01 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.573238 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577a86-8613-4c2e-a6b7-4724bcecea01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.573401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577a86-8613-4c2e-a6b7-4724bcecea01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.574295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee577a86-8613-4c2e-a6b7-4724bcecea01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:20.582094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee577a86-8613-4c2e-a6b7-4724bcecea01 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:20.598720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1013/2000] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:20.601984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577a86-8613-4c2e-a6b7-4724bcecea01 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:20.617269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577a86-8613-4c2e-a6b7-4724bcecea01 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:20.622132 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc5d19-d74c-4828-b4c4-8f2a0480abfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:20.623073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc5d19-d74c-4828-b4c4-8f2a0480abfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.623206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc5d19-d74c-4828-b4c4-8f2a0480abfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.623436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc5d19-d74c-4828-b4c4-8f2a0480abfa admin admin] PATH_INFO: `/v3/projects/c3c706b6d26b4451bd3951c6b489c38a/users/7abd2d7d7fd747d59b118281f5179fdf/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.640954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc5d19-d74c-4828-b4c4-8f2a0480abfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c706b6d26b4451bd3951c6b489c38a, user_id=7abd2d7d7fd747d59b118281f5179fdf, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:20.644378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc5d19-d74c-4828-b4c4-8f2a0480abfa admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:20.734085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1014/2001] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:20 2022] PUT /identity/v3/projects/c3c706b6d26b4451bd3951c6b489c38a/users/7abd2d7d7fd747d59b118281f5179fdf/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:20.743903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc5d19-d74c-4828-b4c4-8f2a0480abfa admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:20.743903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-56cc5d19-d74c-4828-b4c4-8f2a0480abfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:20.775609 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bbe63442-2a03-4e24-9be0-cf220bc0a15c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e4fdaf1205c4816b5c477edc2097ddb'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:20.788457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bbe63442-2a03-4e24-9be0-cf220bc0a15c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e4fdaf1205c4816b5c477edc2097ddb'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:20.795700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bbe63442-2a03-4e24-9be0-cf220bc0a15c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e4fdaf1205c4816b5c477edc2097ddb'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:20.797480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc5d19-d74c-4828-b4c4-8f2a0480abfa admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:20.801862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a113753-aa2f-44f2-ba53-1890a1fb9e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:20.802473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a113753-aa2f-44f2-ba53-1890a1fb9e8c service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.802793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a113753-aa2f-44f2-ba53-1890a1fb9e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.803070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a113753-aa2f-44f2-ba53-1890a1fb9e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.804802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7a113753-aa2f-44f2-ba53-1890a1fb9e8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:20.827599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:20.865193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bbe63442-2a03-4e24-9be0-cf220bc0a15c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e4fdaf1205c4816b5c477edc2097ddb'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:20.865812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bbe63442-2a03-4e24-9be0-cf220bc0a15c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e4fdaf1205c4816b5c477edc2097ddb'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:20.869484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a113753-aa2f-44f2-ba53-1890a1fb9e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:20.871198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a113753-aa2f-44f2-ba53-1890a1fb9e8c service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:20.876195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 988/2002] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:20 2022] DELETE /identity/v3/users/2e4fdaf1205c4816b5c477edc2097ddb => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:20.876195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bc19f-73bd-4311-a6fb-4fbe7ed91b52 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.876195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bc19f-73bd-4311-a6fb-4fbe7ed91b52 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.876195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bc19f-73bd-4311-a6fb-4fbe7ed91b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.881665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1015/2003] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:20.883196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222c8ff-d401-4298-8336-c998ef774b18 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:20.883676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222c8ff-d401-4298-8336-c998ef774b18 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:20.884022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222c8ff-d401-4298-8336-c998ef774b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:20.903894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8222c8ff-d401-4298-8336-c998ef774b18 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:20.905197 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8222c8ff-d401-4298-8336-c998ef774b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:20.906239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bf2bc19f-73bd-4311-a6fb-4fbe7ed91b52 None None] MFA Rules not processed for user `7abd2d7d7fd747d59b118281f5179fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:20.907526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bf2bc19f-73bd-4311-a6fb-4fbe7ed91b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:20.986379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 989/2004] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:20.990484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1016/2005] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:20.995604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2bc19f-73bd-4311-a6fb-4fbe7ed91b52 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:20.997035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bf2bc19f-73bd-4311-a6fb-4fbe7ed91b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:21.002547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222c8ff-d401-4298-8336-c998ef774b18 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:21.004002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8222c8ff-d401-4298-8336-c998ef774b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:21.011728 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2bc19f-73bd-4311-a6fb-4fbe7ed91b52 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:21.016469 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d09b50-1343-4911-a68c-ee67fea35c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:21.017082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d09b50-1343-4911-a68c-ee67fea35c6c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:21.017472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d09b50-1343-4911-a68c-ee67fea35c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:21.017769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d09b50-1343-4911-a68c-ee67fea35c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:21.018393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d09b50-1343-4911-a68c-ee67fea35c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:21.020886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d09b50-1343-4911-a68c-ee67fea35c6c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:21.067806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222c8ff-d401-4298-8336-c998ef774b18 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:21.071557 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85229369-f570-4182-991a-41b9d9c46716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:21.072169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229369-f570-4182-991a-41b9d9c46716 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:21.072453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229369-f570-4182-991a-41b9d9c46716 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:21.072755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229369-f570-4182-991a-41b9d9c46716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:21.082024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85229369-f570-4182-991a-41b9d9c46716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:21.084146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85229369-f570-4182-991a-41b9d9c46716 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:21.090449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 990/2006] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:20 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:21.091469 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d09b50-1343-4911-a68c-ee67fea35c6c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:21.099126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1017/2007] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:21.155024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d09b50-1343-4911-a68c-ee67fea35c6c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:21.161081 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180da5e-30ff-4f32-a0ce-8796f04c1e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:21.161920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180da5e-30ff-4f32-a0ce-8796f04c1e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:21.162273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180da5e-30ff-4f32-a0ce-8796f04c1e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:21.162618 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180da5e-30ff-4f32-a0ce-8796f04c1e15 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:21.163380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180da5e-30ff-4f32-a0ce-8796f04c1e15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:21.166889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180da5e-30ff-4f32-a0ce-8796f04c1e15 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:21.282329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 991/2008] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:21 2022] POST /identity/v3/users => generated 295 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:21.287208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85229369-f570-4182-991a-41b9d9c46716 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:21.302827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85229369-f570-4182-991a-41b9d9c46716 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:21.313439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6814c5-b704-4169-aa90-1ddb2e3c0f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:21.314223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6814c5-b704-4169-aa90-1ddb2e3c0f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:21.314509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6814c5-b704-4169-aa90-1ddb2e3c0f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:21.314825 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6814c5-b704-4169-aa90-1ddb2e3c0f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:21.315478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6814c5-b704-4169-aa90-1ddb2e3c0f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:21.317152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6814c5-b704-4169-aa90-1ddb2e3c0f6c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:21.327576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1018/2009] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:21.331929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180da5e-30ff-4f32-a0ce-8796f04c1e15 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:21.343349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180da5e-30ff-4f32-a0ce-8796f04c1e15 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:21.348906 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ff74d-9985-44b5-b92c-0ecff85b4ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:21.349692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ff74d-9985-44b5-b92c-0ecff85b4ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:21.350568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ff74d-9985-44b5-b92c-0ecff85b4ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:21.350882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ff74d-9985-44b5-b92c-0ecff85b4ea6 admin admin] PATH_INFO: `/v3/projects/b2d36f7b0d684ba191e0b8c71b2726dc/users/dc35e589a7514c94ae31a60b576e150c/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:21.369129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ff74d-9985-44b5-b92c-0ecff85b4ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d36f7b0d684ba191e0b8c71b2726dc, user_id=dc35e589a7514c94ae31a60b576e150c, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:21.372068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ff74d-9985-44b5-b92c-0ecff85b4ea6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:21.405456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:21.439352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 992/2010] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:21 2022] PUT /identity/v3/projects/b2d36f7b0d684ba191e0b8c71b2726dc/users/dc35e589a7514c94ae31a60b576e150c/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:21.443981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6814c5-b704-4169-aa90-1ddb2e3c0f6c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:21.458393 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6814c5-b704-4169-aa90-1ddb2e3c0f6c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:21.518350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd677f-c09f-4427-884c-507bcbb3200c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:21.519016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd677f-c09f-4427-884c-507bcbb3200c admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:21.519477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd677f-c09f-4427-884c-507bcbb3200c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:21.519821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd677f-c09f-4427-884c-507bcbb3200c admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:21.520910 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd677f-c09f-4427-884c-507bcbb3200c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:21.522473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd677f-c09f-4427-884c-507bcbb3200c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:21.536291 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1019/2011] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:21.541407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ff74d-9985-44b5-b92c-0ecff85b4ea6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:21.551962 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ff74d-9985-44b5-b92c-0ecff85b4ea6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:21.556637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860983e-f427-477b-8c99-974cd728f83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:21.557291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860983e-f427-477b-8c99-974cd728f83c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:21.557572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860983e-f427-477b-8c99-974cd728f83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:21.557847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860983e-f427-477b-8c99-974cd728f83c admin admin] PATH_INFO: `/v3/projects/b2d36f7b0d684ba191e0b8c71b2726dc/users/dc35e589a7514c94ae31a60b576e150c/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:21.559768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860983e-f427-477b-8c99-974cd728f83c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d36f7b0d684ba191e0b8c71b2726dc, user_id=dc35e589a7514c94ae31a60b576e150c, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:21.562471 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860983e-f427-477b-8c99-974cd728f83c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:21.572989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 993/2012] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:21 2022] PUT /identity/v3/projects/b2d36f7b0d684ba191e0b8c71b2726dc/users/dc35e589a7514c94ae31a60b576e150c/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:21.584110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663128a-e030-4968-864a-8bd30678f420 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:21.584110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663128a-e030-4968-864a-8bd30678f420 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:21.584307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663128a-e030-4968-864a-8bd30678f420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:21.615666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b663128a-e030-4968-864a-8bd30678f420 None None] MFA Rules not processed for user `dc35e589a7514c94ae31a60b576e150c`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:21.616768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b663128a-e030-4968-864a-8bd30678f420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:21.626762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860983e-f427-477b-8c99-974cd728f83c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:21.630584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a860983e-f427-477b-8c99-974cd728f83c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:21.708377 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1020/2013] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:21.722886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860983e-f427-477b-8c99-974cd728f83c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:21.732631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:21.732631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:21.732631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:21.732631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] PATH_INFO: `/v3/projects/e4834019a5e44a0388220402d877e997` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:21.733008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4834019a5e44a0388220402d877e997)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:21.733008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:21.852804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4834019a5e44a0388220402d877e997', 'request_id': 'req-139574d6-68d1-4666-b970-ec7609635543'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:21.859138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4834019a5e44a0388220402d877e997', 'request_id': 'req-139574d6-68d1-4666-b970-ec7609635543'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:21.865195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4834019a5e44a0388220402d877e997', 'request_id': 'req-139574d6-68d1-4666-b970-ec7609635543'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:21.953306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] The token cache is being invalidate because project e4834019a5e44a0388220402d877e997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:21.953618 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:21.954274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-139574d6-68d1-4666-b970-ec7609635543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:21.956328 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 994/2014] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:21 2022] DELETE /identity/v3/projects/e4834019a5e44a0388220402d877e997 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:21.961720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bce0b-7191-44f5-bf5c-8e75a713161a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:21.962215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bce0b-7191-44f5-bf5c-8e75a713161a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:21.962303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bce0b-7191-44f5-bf5c-8e75a713161a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:21.982302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-538bce0b-7191-44f5-bf5c-8e75a713161a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:21.983601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-538bce0b-7191-44f5-bf5c-8e75a713161a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:22.041895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1021/2015] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:22.049587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6226ab-98cb-492f-9316-a37e01a986b2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.049878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6226ab-98cb-492f-9316-a37e01a986b2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.050298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6226ab-98cb-492f-9316-a37e01a986b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.071440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ff6226ab-98cb-492f-9316-a37e01a986b2 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:22.072674 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ff6226ab-98cb-492f-9316-a37e01a986b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:22.080831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 995/2016] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:22.085405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538bce0b-7191-44f5-bf5c-8e75a713161a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:22.094910 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538bce0b-7191-44f5-bf5c-8e75a713161a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:22.100201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8dde3-0d6e-4aab-9c41-496af3da1fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:22.100808 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8dde3-0d6e-4aab-9c41-496af3da1fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.101087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8dde3-0d6e-4aab-9c41-496af3da1fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.101361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8dde3-0d6e-4aab-9c41-496af3da1fd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.102014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8dde3-0d6e-4aab-9c41-496af3da1fd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:22.102319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8dde3-0d6e-4aab-9c41-496af3da1fd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:22.104085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8dde3-0d6e-4aab-9c41-496af3da1fd8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:22.116213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1022/2017] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:06:22.120611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6226ab-98cb-492f-9316-a37e01a986b2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:22.130504 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6226ab-98cb-492f-9316-a37e01a986b2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:22.135331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906cf6c-6411-4ad0-b455-1c8ec0a448df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:22.135946 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906cf6c-6411-4ad0-b455-1c8ec0a448df admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.136231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906cf6c-6411-4ad0-b455-1c8ec0a448df admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.136508 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906cf6c-6411-4ad0-b455-1c8ec0a448df admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.137155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906cf6c-6411-4ad0-b455-1c8ec0a448df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:22.139095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906cf6c-6411-4ad0-b455-1c8ec0a448df admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:22.202241 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8dde3-0d6e-4aab-9c41-496af3da1fd8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:22.205297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-85f8dde3-0d6e-4aab-9c41-496af3da1fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:22.218053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 996/2018] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:22 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:22.222162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906cf6c-6411-4ad0-b455-1c8ec0a448df admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:22.263487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8dde3-0d6e-4aab-9c41-496af3da1fd8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:22.283632 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906cf6c-6411-4ad0-b455-1c8ec0a448df admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:22.295572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412ad49-cdd2-4fb6-ad69-cf492dffe991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:22.296225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412ad49-cdd2-4fb6-ad69-cf492dffe991 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.296530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412ad49-cdd2-4fb6-ad69-cf492dffe991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.296972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412ad49-cdd2-4fb6-ad69-cf492dffe991 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.297615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412ad49-cdd2-4fb6-ad69-cf492dffe991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:22.299589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412ad49-cdd2-4fb6-ad69-cf492dffe991 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:22.321405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081eb28-0cf8-4e93-867a-b7a0d227a98a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:22.322168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081eb28-0cf8-4e93-867a-b7a0d227a98a service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.322519 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081eb28-0cf8-4e93-867a-b7a0d227a98a service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.323179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081eb28-0cf8-4e93-867a-b7a0d227a98a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.325239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f081eb28-0cf8-4e93-867a-b7a0d227a98a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:22.345281 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:22.381564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 997/2019] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:22 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:22.382700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412ad49-cdd2-4fb6-ad69-cf492dffe991 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:22.394876 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412ad49-cdd2-4fb6-ad69-cf492dffe991 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:22.403283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8a4a7-601d-422f-9650-7c47dfb7be36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:22.403907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8a4a7-601d-422f-9650-7c47dfb7be36 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.404190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8a4a7-601d-422f-9650-7c47dfb7be36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.404463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8a4a7-601d-422f-9650-7c47dfb7be36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.405112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8a4a7-601d-422f-9650-7c47dfb7be36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:22.407618 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8a4a7-601d-422f-9650-7c47dfb7be36 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:22.420266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081eb28-0cf8-4e93-867a-b7a0d227a98a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:22.422366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081eb28-0cf8-4e93-867a-b7a0d227a98a service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:22.434723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 998/2020] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:22.438366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1023/2021] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:22.439005 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8a4a7-601d-422f-9650-7c47dfb7be36 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:22.449781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8a4a7-601d-422f-9650-7c47dfb7be36 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:22.454756 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68ea13-0e62-4802-b677-88e552b40874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:22.455289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68ea13-0e62-4802-b677-88e552b40874 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.455722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68ea13-0e62-4802-b677-88e552b40874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.455999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68ea13-0e62-4802-b677-88e552b40874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.456639 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68ea13-0e62-4802-b677-88e552b40874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:22.458310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68ea13-0e62-4802-b677-88e552b40874 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:22.473765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 999/2022] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:22.477377 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081eb28-0cf8-4e93-867a-b7a0d227a98a service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:22.498595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081eb28-0cf8-4e93-867a-b7a0d227a98a service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:22.498854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68ea13-0e62-4802-b677-88e552b40874 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:22.500479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2d68ea13-0e62-4802-b677-88e552b40874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:22.506113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7b9d6-7c50-41a5-9e7a-74125fbf5b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:22.506988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7b9d6-7c50-41a5-9e7a-74125fbf5b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.507335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7b9d6-7c50-41a5-9e7a-74125fbf5b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.507714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7b9d6-7c50-41a5-9e7a-74125fbf5b95 admin admin] PATH_INFO: `/v3/projects/90cd8bc81fff4256877db5ac6b9df4ea/users/7ae69073291a4d279d213cd5d3147236/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.513519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68ea13-0e62-4802-b677-88e552b40874 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:22.518035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffca66a-0175-43cf-842d-f4c07dbf6d20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:22.518718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffca66a-0175-43cf-842d-f4c07dbf6d20 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.519008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffca66a-0175-43cf-842d-f4c07dbf6d20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.519280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffca66a-0175-43cf-842d-f4c07dbf6d20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.521787 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5ffca66a-0175-43cf-842d-f4c07dbf6d20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:22.545375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7b9d6-7c50-41a5-9e7a-74125fbf5b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90cd8bc81fff4256877db5ac6b9df4ea, user_id=7ae69073291a4d279d213cd5d3147236, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:22.549152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7b9d6-7c50-41a5-9e7a-74125fbf5b95 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:22.595457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffca66a-0175-43cf-842d-f4c07dbf6d20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:22.596906 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffca66a-0175-43cf-842d-f4c07dbf6d20 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:22.606993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1000/2023] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:22.637958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1024/2024] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:22 2022] PUT /identity/v3/projects/90cd8bc81fff4256877db5ac6b9df4ea/users/7ae69073291a4d279d213cd5d3147236/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:22.655234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81288ca-120a-4f87-8b89-b6f7c19ecb9a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.655521 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81288ca-120a-4f87-8b89-b6f7c19ecb9a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.655797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81288ca-120a-4f87-8b89-b6f7c19ecb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.684527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f81288ca-120a-4f87-8b89-b6f7c19ecb9a None None] MFA Rules not processed for user `7ae69073291a4d279d213cd5d3147236`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:22.685810 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f81288ca-120a-4f87-8b89-b6f7c19ecb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:22.693290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7b9d6-7c50-41a5-9e7a-74125fbf5b95 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:22.756594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1001/2025] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:22.756929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7b9d6-7c50-41a5-9e7a-74125fbf5b95 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:22.761110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163b322-adc4-4f7f-9abd-0097029f14ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:22.766439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163b322-adc4-4f7f-9abd-0097029f14ce service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.766439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163b322-adc4-4f7f-9abd-0097029f14ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.766439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163b322-adc4-4f7f-9abd-0097029f14ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.766439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5163b322-adc4-4f7f-9abd-0097029f14ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:22.807224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81288ca-120a-4f87-8b89-b6f7c19ecb9a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:22.814056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:22.817130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81288ca-120a-4f87-8b89-b6f7c19ecb9a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:22.820800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9efcfd-1af5-48b7-866f-c2d2f3e946c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:22.821398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9efcfd-1af5-48b7-866f-c2d2f3e946c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:22.821676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9efcfd-1af5-48b7-866f-c2d2f3e946c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:22.821948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9efcfd-1af5-48b7-866f-c2d2f3e946c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:22.827225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163b322-adc4-4f7f-9abd-0097029f14ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:22.828667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163b322-adc4-4f7f-9abd-0097029f14ce service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:22.830496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9efcfd-1af5-48b7-866f-c2d2f3e946c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:22.832096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9efcfd-1af5-48b7-866f-c2d2f3e946c8 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:22.838047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1025/2026] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:22.846421 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1002/2027] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:23.165109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:23.247981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:23.522043 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:23.691323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:23.833788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:24.093159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:34.001416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:34.178804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=42 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:38.506543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163b322-adc4-4f7f-9abd-0097029f14ce service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:38.508017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5163b322-adc4-4f7f-9abd-0097029f14ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:38.578260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163b322-adc4-4f7f-9abd-0097029f14ce service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:38.583288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:38.584653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:38.584992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:38.585639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] PATH_INFO: `/v3/users/28e0983c384c47ca9ac2f04149956753` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:38.586884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e0983c384c47ca9ac2f04149956753)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:38.588865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:38.752762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e0983c384c47ca9ac2f04149956753'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:38.765652 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e0983c384c47ca9ac2f04149956753'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:38.773314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e0983c384c47ca9ac2f04149956753'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:38.850387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e0983c384c47ca9ac2f04149956753'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:38.850999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e0983c384c47ca9ac2f04149956753'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:38.857624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1026/2028] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:38 2022] DELETE /identity/v3/users/28e0983c384c47ca9ac2f04149956753 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:38.862063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18caf-0bdb-4fc4-bf68-5e60007ffd1c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:38.862552 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18caf-0bdb-4fc4-bf68-5e60007ffd1c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:38.862719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18caf-0bdb-4fc4-bf68-5e60007ffd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:38.882104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-72e18caf-0bdb-4fc4-bf68-5e60007ffd1c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:38.883378 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-72e18caf-0bdb-4fc4-bf68-5e60007ffd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:38.942530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1003/2029] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:38.953421 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:38.955004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:39.019261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7accfae-0f81-4e7e-aae0-048b439d8f14 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:39.023002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dc6a8-ba47-4250-9ece-aa26585ea10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:39.023611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dc6a8-ba47-4250-9ece-aa26585ea10f service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:39.023892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dc6a8-ba47-4250-9ece-aa26585ea10f service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:39.024165 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dc6a8-ba47-4250-9ece-aa26585ea10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:39.033293 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dc6a8-ba47-4250-9ece-aa26585ea10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:39.035023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dc6a8-ba47-4250-9ece-aa26585ea10f service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:39.044686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1027/2030] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:39.563361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e18caf-0bdb-4fc4-bf68-5e60007ffd1c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:39.564839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-72e18caf-0bdb-4fc4-bf68-5e60007ffd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:39.584777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e18caf-0bdb-4fc4-bf68-5e60007ffd1c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:39.590319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:39.591004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:39.591289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:39.591648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] PATH_INFO: `/v3/projects/bb39ded310e3409184bc4a8adb1a9a40` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:39.602710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb39ded310e3409184bc4a8adb1a9a40)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:39.602710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:39.780216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb39ded310e3409184bc4a8adb1a9a40', 'request_id': 'req-26e5c2eb-831b-469e-be5a-59390d2a3cf5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:39.793172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb39ded310e3409184bc4a8adb1a9a40', 'request_id': 'req-26e5c2eb-831b-469e-be5a-59390d2a3cf5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:39.797470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb39ded310e3409184bc4a8adb1a9a40', 'request_id': 'req-26e5c2eb-831b-469e-be5a-59390d2a3cf5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:39.869721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] The token cache is being invalidate because project bb39ded310e3409184bc4a8adb1a9a40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:39.870303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:39.870690 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:39.872218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1004/2031] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:39 2022] DELETE /identity/v3/projects/bb39ded310e3409184bc4a8adb1a9a40 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:39.876234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dc6a8-ba47-4250-9ece-aa26585ea10f service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:39.885932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dc6a8-ba47-4250-9ece-aa26585ea10f service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:39.904884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:39.943476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:39.944151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:39.944376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:39.944644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] PATH_INFO: `/v3/users/e7b4fada64d047bb83dd9a771f20b3ec` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:39.946003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7b4fada64d047bb83dd9a771f20b3ec)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:39.947949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:40.332434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7b4fada64d047bb83dd9a771f20b3ec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:40.344516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7b4fada64d047bb83dd9a771f20b3ec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:40.351406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7b4fada64d047bb83dd9a771f20b3ec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:40.432787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7b4fada64d047bb83dd9a771f20b3ec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:40.433480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7b4fada64d047bb83dd9a771f20b3ec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:40.440568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1028/2032] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:39 2022] DELETE /identity/v3/users/e7b4fada64d047bb83dd9a771f20b3ec => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:40.861982 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:40.864109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:40.880384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c2eb-831b-469e-be5a-59390d2a3cf5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:40.951285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:40.951285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:40.951285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:40.951285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] PATH_INFO: `/v3/projects/c2f4c815d18148319447c08261151ce9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:40.951681 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f4c815d18148319447c08261151ce9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:40.953524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:40.996480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:40.998097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:41.011506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce3cb1-63c8-442f-a6e5-3f326f973ac5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:41.016553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de670642-5b4f-4acc-89b5-4293bddc57bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:41.017194 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de670642-5b4f-4acc-89b5-4293bddc57bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:41.017500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de670642-5b4f-4acc-89b5-4293bddc57bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:41.017764 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de670642-5b4f-4acc-89b5-4293bddc57bc admin admin] PATH_INFO: `/v3/users/7abd2d7d7fd747d59b118281f5179fdf` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:41.019284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de670642-5b4f-4acc-89b5-4293bddc57bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7abd2d7d7fd747d59b118281f5179fdf)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:41.021331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de670642-5b4f-4acc-89b5-4293bddc57bc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:41.053073 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:41.070600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f4c815d18148319447c08261151ce9', 'request_id': 'req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.077864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f4c815d18148319447c08261151ce9', 'request_id': 'req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.083465 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f4c815d18148319447c08261151ce9', 'request_id': 'req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.190904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] The token cache is being invalidate because project c2f4c815d18148319447c08261151ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:41.191433 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.192231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.194389 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1005/2033] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:40 2022] DELETE /identity/v3/projects/c2f4c815d18148319447c08261151ce9 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:41.198808 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:41.200925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:41.281269 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f6e90-1b8f-472d-a7f0-2acea66dfdaf admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:41.287014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-de670642-5b4f-4acc-89b5-4293bddc57bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7abd2d7d7fd747d59b118281f5179fdf'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.287622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:41.288277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:41.288953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:41.289301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] PATH_INFO: `/v3/users/eb815a3d46bb447985cd021def5dccb1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:41.291148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb815a3d46bb447985cd021def5dccb1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:41.293798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:41.306507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-de670642-5b4f-4acc-89b5-4293bddc57bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7abd2d7d7fd747d59b118281f5179fdf'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.330237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-de670642-5b4f-4acc-89b5-4293bddc57bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7abd2d7d7fd747d59b118281f5179fdf'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.414477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-de670642-5b4f-4acc-89b5-4293bddc57bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7abd2d7d7fd747d59b118281f5179fdf'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.415312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-de670642-5b4f-4acc-89b5-4293bddc57bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7abd2d7d7fd747d59b118281f5179fdf'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.424473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1029/2034] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:40 2022] DELETE /identity/v3/users/7abd2d7d7fd747d59b118281f5179fdf => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:41.428963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eff81-ad85-4233-8457-62c13394cc34 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:41.429400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eff81-ad85-4233-8457-62c13394cc34 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:41.429684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eff81-ad85-4233-8457-62c13394cc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:41.460569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-df9eff81-ad85-4233-8457-62c13394cc34 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:41.461904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-df9eff81-ad85-4233-8457-62c13394cc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:41.471740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:41.565682 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb815a3d46bb447985cd021def5dccb1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.591536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1030/2035] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:41.602926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb815a3d46bb447985cd021def5dccb1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.606669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb815a3d46bb447985cd021def5dccb1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.623103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eff81-ad85-4233-8457-62c13394cc34 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:41.624527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-df9eff81-ad85-4233-8457-62c13394cc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:41.681754 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb815a3d46bb447985cd021def5dccb1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.682470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb815a3d46bb447985cd021def5dccb1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:41.691909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1006/2036] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:41 2022] DELETE /identity/v3/users/eb815a3d46bb447985cd021def5dccb1 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:41.712872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eff81-ad85-4233-8457-62c13394cc34 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:41.787958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89017411-e2c4-4dff-aff1-fd9de507cb71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:41.789763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017411-e2c4-4dff-aff1-fd9de507cb71 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:41.789763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017411-e2c4-4dff-aff1-fd9de507cb71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:41.789763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017411-e2c4-4dff-aff1-fd9de507cb71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:41.791161 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-89017411-e2c4-4dff-aff1-fd9de507cb71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:41.860815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89017411-e2c4-4dff-aff1-fd9de507cb71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:41.862359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89017411-e2c4-4dff-aff1-fd9de507cb71 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:41.874763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1031/2037] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:42.423170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:42.424642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:42.441177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16342ac-41a3-4172-818c-202c9ddfbba0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:42.448034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:42.448845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:42.451582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:42.451582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] PATH_INFO: `/v3/projects/3bd5bd1ddfef47919f0f3e37b790b746` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:42.451582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd5bd1ddfef47919f0f3e37b790b746)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:42.452599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:42.546926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:42.551593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89017411-e2c4-4dff-aff1-fd9de507cb71 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:42.554979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-89017411-e2c4-4dff-aff1-fd9de507cb71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:42.566105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89017411-e2c4-4dff-aff1-fd9de507cb71 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:42.569104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd5bd1ddfef47919f0f3e37b790b746', 'request_id': 'req-67a11b19-9a37-4b27-90fd-3d586d28eacb'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:42.587758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd5bd1ddfef47919f0f3e37b790b746', 'request_id': 'req-67a11b19-9a37-4b27-90fd-3d586d28eacb'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:42.590778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:42.597087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd5bd1ddfef47919f0f3e37b790b746', 'request_id': 'req-67a11b19-9a37-4b27-90fd-3d586d28eacb'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:42.644798 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:42.645598 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:42.646166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:42.646467 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] PATH_INFO: `/v3/projects/c3c706b6d26b4451bd3951c6b489c38a` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:42.647702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c706b6d26b4451bd3951c6b489c38a)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:42.650163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:42.784329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] The token cache is being invalidate because project 3bd5bd1ddfef47919f0f3e37b790b746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:42.788444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:42.788444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67a11b19-9a37-4b27-90fd-3d586d28eacb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:42.789916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1007/2038] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:42 2022] DELETE /identity/v3/projects/3bd5bd1ddfef47919f0f3e37b790b746 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:42.795302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ef917-5e78-41a2-a54a-5fc25382fd1b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:42.795786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ef917-5e78-41a2-a54a-5fc25382fd1b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:42.796081 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ef917-5e78-41a2-a54a-5fc25382fd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:42.823016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-377ef917-5e78-41a2-a54a-5fc25382fd1b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:42.824054 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-377ef917-5e78-41a2-a54a-5fc25382fd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:42.846994 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1008/2039] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:42.856444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8861692-26ba-42d3-8fd5-7da4b187b5d3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:42.857082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8861692-26ba-42d3-8fd5-7da4b187b5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:42.857365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8861692-26ba-42d3-8fd5-7da4b187b5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:42.890729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b8861692-26ba-42d3-8fd5-7da4b187b5d3 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:42.890729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b8861692-26ba-42d3-8fd5-7da4b187b5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:42.906843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c706b6d26b4451bd3951c6b489c38a', 'request_id': 'req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:42.918930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c706b6d26b4451bd3951c6b489c38a', 'request_id': 'req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:42.926481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c706b6d26b4451bd3951c6b489c38a', 'request_id': 'req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:42.964391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1009/2040] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:42.971085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8861692-26ba-42d3-8fd5-7da4b187b5d3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:42.981221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8861692-26ba-42d3-8fd5-7da4b187b5d3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:42.987180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280a434-a93a-4f85-9579-eb9b37e182f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:42.987777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280a434-a93a-4f85-9579-eb9b37e182f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:42.988119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280a434-a93a-4f85-9579-eb9b37e182f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:42.988511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280a434-a93a-4f85-9579-eb9b37e182f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:42.989348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280a434-a93a-4f85-9579-eb9b37e182f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:42.989724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280a434-a93a-4f85-9579-eb9b37e182f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:42.991467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280a434-a93a-4f85-9579-eb9b37e182f2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:43.010885 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1010/2041] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:06:43.012548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] The token cache is being invalidate because project c3c706b6d26b4451bd3951c6b489c38a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:43.012680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280a434-a93a-4f85-9579-eb9b37e182f2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:43.012965 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:43.013639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:43.016680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1032/2042] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:42 2022] DELETE /identity/v3/projects/c3c706b6d26b4451bd3951c6b489c38a => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:43.021575 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:43.023241 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:43.027152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280a434-a93a-4f85-9579-eb9b37e182f2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:43.041850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb2fcb-2a24-4fe8-82b5-e171debd1046 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:43.047885 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa0dbf-9028-4d13-9999-83e6ba76dbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:43.048464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa0dbf-9028-4d13-9999-83e6ba76dbaf admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:43.048755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa0dbf-9028-4d13-9999-83e6ba76dbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:43.049013 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa0dbf-9028-4d13-9999-83e6ba76dbaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:43.049667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa0dbf-9028-4d13-9999-83e6ba76dbaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:43.051856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa0dbf-9028-4d13-9999-83e6ba76dbaf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:43.053581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4a05b-04af-455a-8d41-524d07be97a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:43.054249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4a05b-04af-455a-8d41-524d07be97a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:43.054571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4a05b-04af-455a-8d41-524d07be97a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:43.054928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4a05b-04af-455a-8d41-524d07be97a5 admin admin] PATH_INFO: `/v3/users/dc35e589a7514c94ae31a60b576e150c` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:43.056276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4a05b-04af-455a-8d41-524d07be97a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc35e589a7514c94ae31a60b576e150c)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:43.058260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4a05b-04af-455a-8d41-524d07be97a5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:43.198670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1011/2043] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:43 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:43.199591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa0dbf-9028-4d13-9999-83e6ba76dbaf admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:43.201463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9faa0dbf-9028-4d13-9999-83e6ba76dbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:43.278229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa0dbf-9028-4d13-9999-83e6ba76dbaf admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:43.282950 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dd52a-58c0-4128-a276-7f16adf3a12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:43.283558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dd52a-58c0-4128-a276-7f16adf3a12b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:43.284146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dd52a-58c0-4128-a276-7f16adf3a12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:43.284420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dd52a-58c0-4128-a276-7f16adf3a12b admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:43.285086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dd52a-58c0-4128-a276-7f16adf3a12b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:43.287103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dd52a-58c0-4128-a276-7f16adf3a12b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:43.340419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31a4a05b-04af-455a-8d41-524d07be97a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc35e589a7514c94ae31a60b576e150c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:43.356879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31a4a05b-04af-455a-8d41-524d07be97a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc35e589a7514c94ae31a60b576e150c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:43.365603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31a4a05b-04af-455a-8d41-524d07be97a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc35e589a7514c94ae31a60b576e150c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:43.405418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1012/2044] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:43 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:43.409622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dd52a-58c0-4128-a276-7f16adf3a12b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:43.421496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dd52a-58c0-4128-a276-7f16adf3a12b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:43.426374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7193211-1e24-4b14-a098-24f59b284cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:43.427745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7193211-1e24-4b14-a098-24f59b284cdd admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:43.428373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7193211-1e24-4b14-a098-24f59b284cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:43.428804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7193211-1e24-4b14-a098-24f59b284cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:43.429617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7193211-1e24-4b14-a098-24f59b284cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:43.431429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7193211-1e24-4b14-a098-24f59b284cdd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:43.444965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1013/2045] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:43.449258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7193211-1e24-4b14-a098-24f59b284cdd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:43.460998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7193211-1e24-4b14-a098-24f59b284cdd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:43.466221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf42ad4-2352-4707-b65d-3210b3ab7082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:43.466871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42ad4-2352-4707-b65d-3210b3ab7082 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:43.467282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42ad4-2352-4707-b65d-3210b3ab7082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:43.467555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42ad4-2352-4707-b65d-3210b3ab7082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:43.468196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf42ad4-2352-4707-b65d-3210b3ab7082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:43.469789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf42ad4-2352-4707-b65d-3210b3ab7082 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:43.478599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31a4a05b-04af-455a-8d41-524d07be97a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc35e589a7514c94ae31a60b576e150c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:43.479271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31a4a05b-04af-455a-8d41-524d07be97a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc35e589a7514c94ae31a60b576e150c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:43.488209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1014/2046] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:43.488399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1033/2047] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:43 2022] DELETE /identity/v3/users/dc35e589a7514c94ae31a60b576e150c => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:43.489327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf42ad4-2352-4707-b65d-3210b3ab7082 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:43.492392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-faf42ad4-2352-4707-b65d-3210b3ab7082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:43.505846 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf42ad4-2352-4707-b65d-3210b3ab7082 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:43.570439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64b5e0-2496-4ac1-ac24-5d2ae33768cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:43.571142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64b5e0-2496-4ac1-ac24-5d2ae33768cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:43.571419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64b5e0-2496-4ac1-ac24-5d2ae33768cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:43.571695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64b5e0-2496-4ac1-ac24-5d2ae33768cf admin admin] PATH_INFO: `/v3/projects/5870af1bbad64b138178c47e4a571d81/users/66702831c1ca4158976920f6be2cc80c/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:43.588786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64b5e0-2496-4ac1-ac24-5d2ae33768cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5870af1bbad64b138178c47e4a571d81, user_id=66702831c1ca4158976920f6be2cc80c, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:43.591840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64b5e0-2496-4ac1-ac24-5d2ae33768cf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:43.689475 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1015/2048] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:43 2022] PUT /identity/v3/projects/5870af1bbad64b138178c47e4a571d81/users/66702831c1ca4158976920f6be2cc80c/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:43.701127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391c7ae-3a70-497d-9198-d82b57466cb3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:43.701127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391c7ae-3a70-497d-9198-d82b57466cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:43.701127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391c7ae-3a70-497d-9198-d82b57466cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:43.726432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b391c7ae-3a70-497d-9198-d82b57466cb3 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:43.728120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b391c7ae-3a70-497d-9198-d82b57466cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:43.750782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:43.807610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1034/2049] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:43.821143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64b5e0-2496-4ac1-ac24-5d2ae33768cf admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:43.823182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ac64b5e0-2496-4ac1-ac24-5d2ae33768cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:43.883591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:43.897415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64b5e0-2496-4ac1-ac24-5d2ae33768cf admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:43.904290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b09d8-ace9-4abb-b230-f66e034ca299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:43.905314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b09d8-ace9-4abb-b230-f66e034ca299 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:43.905535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b09d8-ace9-4abb-b230-f66e034ca299 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:43.905865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b09d8-ace9-4abb-b230-f66e034ca299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:43.917981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b09d8-ace9-4abb-b230-f66e034ca299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:43.919968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b09d8-ace9-4abb-b230-f66e034ca299 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:43.934335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1016/2050] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:44.139269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391c7ae-3a70-497d-9198-d82b57466cb3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:44.141361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b391c7ae-3a70-497d-9198-d82b57466cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:44.209044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391c7ae-3a70-497d-9198-d82b57466cb3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:44.220975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:44.221634 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:44.221925 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:44.222225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] PATH_INFO: `/v3/projects/b2d36f7b0d684ba191e0b8c71b2726dc` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:44.224629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2d36f7b0d684ba191e0b8c71b2726dc)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:44.236727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:44.361866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d36f7b0d684ba191e0b8c71b2726dc', 'request_id': 'req-ca944587-54d0-4e5a-b37e-d1875c460584'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:44.369710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d36f7b0d684ba191e0b8c71b2726dc', 'request_id': 'req-ca944587-54d0-4e5a-b37e-d1875c460584'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:44.377787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d36f7b0d684ba191e0b8c71b2726dc', 'request_id': 'req-ca944587-54d0-4e5a-b37e-d1875c460584'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:44.484077 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] The token cache is being invalidate because project b2d36f7b0d684ba191e0b8c71b2726dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:44.484365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:44.485013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ca944587-54d0-4e5a-b37e-d1875c460584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:44.486481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1035/2051] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:44 2022] DELETE /identity/v3/projects/b2d36f7b0d684ba191e0b8c71b2726dc => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:44.491487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99497143-72d1-4613-be07-d254ada57fbd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:44.491977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99497143-72d1-4613-be07-d254ada57fbd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:44.492072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99497143-72d1-4613-be07-d254ada57fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:44.512685 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-99497143-72d1-4613-be07-d254ada57fbd None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:44.514735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-99497143-72d1-4613-be07-d254ada57fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:44.586262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1017/2052] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:44.594002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec034d-1663-4d6e-9e46-54967109eff0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:44.594285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec034d-1663-4d6e-9e46-54967109eff0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:44.594562 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec034d-1663-4d6e-9e46-54967109eff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:44.623576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e8ec034d-1663-4d6e-9e46-54967109eff0 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:44.625108 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e8ec034d-1663-4d6e-9e46-54967109eff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:44.649291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99497143-72d1-4613-be07-d254ada57fbd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:44.656075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1036/2053] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:44.661500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99497143-72d1-4613-be07-d254ada57fbd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:44.667672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd374c-36d0-401a-a032-0b93a72f6767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:44.675986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd374c-36d0-401a-a032-0b93a72f6767 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:44.675986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd374c-36d0-401a-a032-0b93a72f6767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:44.675986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd374c-36d0-401a-a032-0b93a72f6767 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:44.675986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd374c-36d0-401a-a032-0b93a72f6767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:44.675986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd374c-36d0-401a-a032-0b93a72f6767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:44.675986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd374c-36d0-401a-a032-0b93a72f6767 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:44.688546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1018/2054] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:06:44.691044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec034d-1663-4d6e-9e46-54967109eff0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:44.702028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec034d-1663-4d6e-9e46-54967109eff0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:44.706944 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd96cf5-8912-49ee-925c-ee61298c5381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:44.707549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd96cf5-8912-49ee-925c-ee61298c5381 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:44.707833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd96cf5-8912-49ee-925c-ee61298c5381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:44.708696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd96cf5-8912-49ee-925c-ee61298c5381 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:44.709377 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd96cf5-8912-49ee-925c-ee61298c5381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:44.711432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd96cf5-8912-49ee-925c-ee61298c5381 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:44.835692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1037/2055] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:44 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:44.839910 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd374c-36d0-401a-a032-0b93a72f6767 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:44.850791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd374c-36d0-401a-a032-0b93a72f6767 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:44.929996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c84a1-9999-4658-9a88-d7412dbb387e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:44.930972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c84a1-9999-4658-9a88-d7412dbb387e admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:44.931492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c84a1-9999-4658-9a88-d7412dbb387e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:44.931980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c84a1-9999-4658-9a88-d7412dbb387e admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:44.933358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c84a1-9999-4658-9a88-d7412dbb387e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:44.935828 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c84a1-9999-4658-9a88-d7412dbb387e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:45.037323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1019/2056] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:44 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:45.041720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd96cf5-8912-49ee-925c-ee61298c5381 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:45.061137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd96cf5-8912-49ee-925c-ee61298c5381 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:45.070530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dee70-315a-49fc-b906-cf9093973e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:45.071392 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dee70-315a-49fc-b906-cf9093973e18 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:45.071819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dee70-315a-49fc-b906-cf9093973e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:45.072167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dee70-315a-49fc-b906-cf9093973e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:45.073313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dee70-315a-49fc-b906-cf9093973e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:45.075003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dee70-315a-49fc-b906-cf9093973e18 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:45.087572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1038/2057] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:45.091524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c84a1-9999-4658-9a88-d7412dbb387e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:45.106009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c84a1-9999-4658-9a88-d7412dbb387e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:45.116661 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0bcc2-3d56-4bf8-a729-ea6c85b81b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:45.118088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0bcc2-3d56-4bf8-a729-ea6c85b81b86 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:45.118399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0bcc2-3d56-4bf8-a729-ea6c85b81b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:45.118897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0bcc2-3d56-4bf8-a729-ea6c85b81b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:45.120011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0bcc2-3d56-4bf8-a729-ea6c85b81b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:45.123427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0bcc2-3d56-4bf8-a729-ea6c85b81b86 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:45.138160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1020/2058] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:45.142205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dee70-315a-49fc-b906-cf9093973e18 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:45.145425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:45.154178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dee70-315a-49fc-b906-cf9093973e18 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:45.159601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a734ab3-0afd-487f-b944-8fa045c499ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:45.160569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a734ab3-0afd-487f-b944-8fa045c499ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:45.160864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a734ab3-0afd-487f-b944-8fa045c499ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:45.161149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a734ab3-0afd-487f-b944-8fa045c499ea admin admin] PATH_INFO: `/v3/projects/64e6eebfe897446580cc756a85558f67/users/0648d7f8df634273bad28a7058d6bdba/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:45.179149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a734ab3-0afd-487f-b944-8fa045c499ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e6eebfe897446580cc756a85558f67, user_id=0648d7f8df634273bad28a7058d6bdba, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:45.181934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a734ab3-0afd-487f-b944-8fa045c499ea admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:45.247929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1039/2059] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:45 2022] PUT /identity/v3/projects/64e6eebfe897446580cc756a85558f67/users/0648d7f8df634273bad28a7058d6bdba/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:45.260385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e42d2-f066-48de-8498-00ee0189c282 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:45.260670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e42d2-f066-48de-8498-00ee0189c282 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:45.260944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e42d2-f066-48de-8498-00ee0189c282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:45.296589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3d2e42d2-f066-48de-8498-00ee0189c282 None None] MFA Rules not processed for user `0648d7f8df634273bad28a7058d6bdba`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:45.296589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3d2e42d2-f066-48de-8498-00ee0189c282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:45.372596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1021/2060] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:45.373756 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a734ab3-0afd-487f-b944-8fa045c499ea admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:45.421244 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:45.443585 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a734ab3-0afd-487f-b944-8fa045c499ea admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:45.449467 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f19996-e888-4637-90c3-68c3057b7762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:45.450861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f19996-e888-4637-90c3-68c3057b7762 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:45.455085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f19996-e888-4637-90c3-68c3057b7762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:45.455085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f19996-e888-4637-90c3-68c3057b7762 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:45.455085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f19996-e888-4637-90c3-68c3057b7762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:45.455085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f19996-e888-4637-90c3-68c3057b7762 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:45.557352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1040/2061] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:45 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:45.559192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e42d2-f066-48de-8498-00ee0189c282 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:45.573772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e42d2-f066-48de-8498-00ee0189c282 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:45.649259 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746fab4-6b82-42a0-9459-aaf167c6580d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:45.650564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fab4-6b82-42a0-9459-aaf167c6580d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:45.651198 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fab4-6b82-42a0-9459-aaf167c6580d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:45.651671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fab4-6b82-42a0-9459-aaf167c6580d admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:45.652966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746fab4-6b82-42a0-9459-aaf167c6580d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:45.656257 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746fab4-6b82-42a0-9459-aaf167c6580d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:45.756762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1022/2062] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:45 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:45.758744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f19996-e888-4637-90c3-68c3057b7762 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:45.759523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:45.771124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f19996-e888-4637-90c3-68c3057b7762 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:45.776939 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12296fc8-431f-4f09-8e33-c8a27ed2cfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:45.777659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12296fc8-431f-4f09-8e33-c8a27ed2cfa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:45.778128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12296fc8-431f-4f09-8e33-c8a27ed2cfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:45.778455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12296fc8-431f-4f09-8e33-c8a27ed2cfa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:45.779296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12296fc8-431f-4f09-8e33-c8a27ed2cfa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:45.781022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12296fc8-431f-4f09-8e33-c8a27ed2cfa8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:45.793100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1041/2063] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:45.797282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746fab4-6b82-42a0-9459-aaf167c6580d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:45.810513 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746fab4-6b82-42a0-9459-aaf167c6580d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:45.824313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba2d7a-828c-4d54-84aa-d75641628416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:45.825615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba2d7a-828c-4d54-84aa-d75641628416 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:45.825905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba2d7a-828c-4d54-84aa-d75641628416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:45.826192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba2d7a-828c-4d54-84aa-d75641628416 admin admin] PATH_INFO: `/v3/projects/d46efe8c9af2406e9fcee58c80dfa2c1/users/a2b2c597a5654ad19b43e3de3f1dd56d/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:45.853924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba2d7a-828c-4d54-84aa-d75641628416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d46efe8c9af2406e9fcee58c80dfa2c1, user_id=a2b2c597a5654ad19b43e3de3f1dd56d, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:45.856864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba2d7a-828c-4d54-84aa-d75641628416 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:45.905422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a3da7-3615-4975-b73b-a60c556be498 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:45.906869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a3da7-3615-4975-b73b-a60c556be498 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:45.907161 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a3da7-3615-4975-b73b-a60c556be498 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:45.909122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1042/2064] 10.222.0.59 () {54 vars in 921 bytes} [Fri Sep 16 08:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:06:45.940268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497f017-3a1b-4a51-9663-50cfb2f1c92f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:45.940706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497f017-3a1b-4a51-9663-50cfb2f1c92f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:45.941475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497f017-3a1b-4a51-9663-50cfb2f1c92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:45.971088 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c497f017-3a1b-4a51-9663-50cfb2f1c92f None None] MFA Rules not processed for user `c2dca05dfe8b4af0a02ea93fbe29c556`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:45.975363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c497f017-3a1b-4a51-9663-50cfb2f1c92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:45.979759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1023/2065] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:45 2022] PUT /identity/v3/projects/d46efe8c9af2406e9fcee58c80dfa2c1/users/a2b2c597a5654ad19b43e3de3f1dd56d/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:45.983526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba2d7a-828c-4d54-84aa-d75641628416 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:46.052194 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1043/2066] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Sep 16 08:06:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:46.054303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba2d7a-828c-4d54-84aa-d75641628416 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:46.060192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7350a-a7b1-4505-a7d1-4575342a323c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:46.060815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7350a-a7b1-4505-a7d1-4575342a323c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:46.061099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7350a-a7b1-4505-a7d1-4575342a323c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:46.061378 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7350a-a7b1-4505-a7d1-4575342a323c admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:46.062025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7350a-a7b1-4505-a7d1-4575342a323c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:46.063740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7350a-a7b1-4505-a7d1-4575342a323c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:46.065530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497f017-3a1b-4a51-9663-50cfb2f1c92f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:46.068169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c497f017-3a1b-4a51-9663-50cfb2f1c92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:46.086988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497f017-3a1b-4a51-9663-50cfb2f1c92f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:46.088292 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1024/2067] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:46.093298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c1a9f-818e-4513-afc4-20ddda5d6171 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:46.093616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7350a-a7b1-4505-a7d1-4575342a323c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:46.093917 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c1a9f-818e-4513-afc4-20ddda5d6171 service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:46.094204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c1a9f-818e-4513-afc4-20ddda5d6171 service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:46.094474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c1a9f-818e-4513-afc4-20ddda5d6171 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:46.107521 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7350a-a7b1-4505-a7d1-4575342a323c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:46.109530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c1a9f-818e-4513-afc4-20ddda5d6171 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:46.111673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c1a9f-818e-4513-afc4-20ddda5d6171 service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:46.112468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af87bf-b104-4a13-9e4e-47b98553b14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:46.113196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af87bf-b104-4a13-9e4e-47b98553b14b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:46.113544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af87bf-b104-4a13-9e4e-47b98553b14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:46.113892 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af87bf-b104-4a13-9e4e-47b98553b14b admin admin] PATH_INFO: `/v3/projects/d46efe8c9af2406e9fcee58c80dfa2c1/users/a2b2c597a5654ad19b43e3de3f1dd56d/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:46.116070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af87bf-b104-4a13-9e4e-47b98553b14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d46efe8c9af2406e9fcee58c80dfa2c1, user_id=a2b2c597a5654ad19b43e3de3f1dd56d, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:46.119207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af87bf-b104-4a13-9e4e-47b98553b14b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:46.122443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1044/2068] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:46.123375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c1a9f-818e-4513-afc4-20ddda5d6171 service placement] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:46.124719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f44c1a9f-818e-4513-afc4-20ddda5d6171 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:46.131310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1025/2069] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:46 2022] PUT /identity/v3/projects/d46efe8c9af2406e9fcee58c80dfa2c1/users/a2b2c597a5654ad19b43e3de3f1dd56d/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:46.140049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c1a9f-818e-4513-afc4-20ddda5d6171 service placement] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:46.146371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f33b-e958-4a69-b124-b58d5f5aa799 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:46.146715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f33b-e958-4a69-b124-b58d5f5aa799 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:46.147001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643f33b-e958-4a69-b124-b58d5f5aa799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:46.182234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4643f33b-e958-4a69-b124-b58d5f5aa799 None None] MFA Rules not processed for user `a2b2c597a5654ad19b43e3de3f1dd56d`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:46.184394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4643f33b-e958-4a69-b124-b58d5f5aa799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:46.205881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a28f82-3f33-4e67-9bec-b67728a5588c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:46.206616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a28f82-3f33-4e67-9bec-b67728a5588c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:46.207405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a28f82-3f33-4e67-9bec-b67728a5588c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:46.207761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a28f82-3f33-4e67-9bec-b67728a5588c admin admin] PATH_INFO: `/v3/users/ae3824cef26941278065945a8ee37d32` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:46.210399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a28f82-3f33-4e67-9bec-b67728a5588c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae3824cef26941278065945a8ee37d32)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:46.212889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a28f82-3f33-4e67-9bec-b67728a5588c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:46.288023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1026/2070] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:46.303783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643f33b-e958-4a69-b124-b58d5f5aa799 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:46.305209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4643f33b-e958-4a69-b124-b58d5f5aa799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:46.392641 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643f33b-e958-4a69-b124-b58d5f5aa799 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:46.412856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971b356-b6db-4107-98fb-06397415a685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:46.412856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971b356-b6db-4107-98fb-06397415a685 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:46.412856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971b356-b6db-4107-98fb-06397415a685 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:46.412856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971b356-b6db-4107-98fb-06397415a685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:46.421333 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-68a28f82-3f33-4e67-9bec-b67728a5588c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae3824cef26941278065945a8ee37d32'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:46.425377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971b356-b6db-4107-98fb-06397415a685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:46.427319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971b356-b6db-4107-98fb-06397415a685 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:46.435258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-68a28f82-3f33-4e67-9bec-b67728a5588c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae3824cef26941278065945a8ee37d32'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:46.444214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-68a28f82-3f33-4e67-9bec-b67728a5588c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae3824cef26941278065945a8ee37d32'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:46.446732 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1027/2071] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:46.532860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-68a28f82-3f33-4e67-9bec-b67728a5588c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae3824cef26941278065945a8ee37d32'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:46.534121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-68a28f82-3f33-4e67-9bec-b67728a5588c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae3824cef26941278065945a8ee37d32'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:46.548320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1045/2072] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:46 2022] DELETE /identity/v3/users/ae3824cef26941278065945a8ee37d32 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:46.659462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:46.740612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:46.914131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971b356-b6db-4107-98fb-06397415a685 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:46.927511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971b356-b6db-4107-98fb-06397415a685 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:47.009710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:47.010453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:47.010837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:47.011178 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] PATH_INFO: `/v3/projects/48c466a9c78b46db90b2c979cf885f43` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:47.012441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c466a9c78b46db90b2c979cf885f43)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:47.014960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:47.188527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c466a9c78b46db90b2c979cf885f43', 'request_id': 'req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:47.195749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c466a9c78b46db90b2c979cf885f43', 'request_id': 'req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:47.203947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c466a9c78b46db90b2c979cf885f43', 'request_id': 'req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:47.316112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] The token cache is being invalidate because project 48c466a9c78b46db90b2c979cf885f43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:47.316408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:47.319559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f70e18d4-cfa3-48b9-bbc1-b563fc55baac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:47.319559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1028/2073] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:46 2022] DELETE /identity/v3/projects/48c466a9c78b46db90b2c979cf885f43 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:47.323675 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa0f40-8b3b-4601-a90f-a8e4606ac44e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:47.324645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa0f40-8b3b-4601-a90f-a8e4606ac44e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:47.325123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa0f40-8b3b-4601-a90f-a8e4606ac44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:47.356055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a2aa0f40-8b3b-4601-a90f-a8e4606ac44e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:47.358707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a2aa0f40-8b3b-4601-a90f-a8e4606ac44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:47.510534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1046/2074] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:47.518356 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9c1fb-5215-4ab2-82aa-686cd9d22479 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:47.518650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9c1fb-5215-4ab2-82aa-686cd9d22479 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:47.518951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9c1fb-5215-4ab2-82aa-686cd9d22479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:47.564866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-cfe9c1fb-5215-4ab2-82aa-686cd9d22479 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:47.566695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cfe9c1fb-5215-4ab2-82aa-686cd9d22479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:47.586816 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1029/2075] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:47.588012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa0f40-8b3b-4601-a90f-a8e4606ac44e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:47.599343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa0f40-8b3b-4601-a90f-a8e4606ac44e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:47.604246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05990703-651a-475a-8c34-54ef485679ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:47.604869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05990703-651a-475a-8c34-54ef485679ac admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:47.609827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05990703-651a-475a-8c34-54ef485679ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:47.609827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05990703-651a-475a-8c34-54ef485679ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:47.609827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05990703-651a-475a-8c34-54ef485679ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:47.609827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05990703-651a-475a-8c34-54ef485679ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:47.609827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05990703-651a-475a-8c34-54ef485679ac admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:47.622027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1047/2076] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:06:47.626754 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9c1fb-5215-4ab2-82aa-686cd9d22479 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:47.640546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9c1fb-5215-4ab2-82aa-686cd9d22479 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:47.647580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76254729-3ecc-4d84-adcc-61b9a62ce30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:47.648203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76254729-3ecc-4d84-adcc-61b9a62ce30d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:47.648482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76254729-3ecc-4d84-adcc-61b9a62ce30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:47.648748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76254729-3ecc-4d84-adcc-61b9a62ce30d admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:47.649384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76254729-3ecc-4d84-adcc-61b9a62ce30d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:47.651444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76254729-3ecc-4d84-adcc-61b9a62ce30d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:47.752012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1030/2077] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:47 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:47.757016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05990703-651a-475a-8c34-54ef485679ac admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:47.768044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05990703-651a-475a-8c34-54ef485679ac admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:47.836020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255780f-9c57-4be9-b1dc-a23728bc808a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:47.836662 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255780f-9c57-4be9-b1dc-a23728bc808a admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:47.837353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255780f-9c57-4be9-b1dc-a23728bc808a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:47.837795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255780f-9c57-4be9-b1dc-a23728bc808a admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:47.838786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255780f-9c57-4be9-b1dc-a23728bc808a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:47.840772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255780f-9c57-4be9-b1dc-a23728bc808a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:47.903113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76254729-3ecc-4d84-adcc-61b9a62ce30d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:47.903113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-76254729-3ecc-4d84-adcc-61b9a62ce30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:47.928726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76254729-3ecc-4d84-adcc-61b9a62ce30d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:47.937811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc9569-517c-4d10-93da-a750ca1c5854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:47.938755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc9569-517c-4d10-93da-a750ca1c5854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:47.939213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc9569-517c-4d10-93da-a750ca1c5854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:47.939663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc9569-517c-4d10-93da-a750ca1c5854 admin admin] PATH_INFO: `/v3/users/0648d7f8df634273bad28a7058d6bdba` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:47.941322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc9569-517c-4d10-93da-a750ca1c5854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0648d7f8df634273bad28a7058d6bdba)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:47.944751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc9569-517c-4d10-93da-a750ca1c5854 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:48.012470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1048/2078] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:47 2022] POST /identity/v3/users => generated 294 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:48.017376 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255780f-9c57-4be9-b1dc-a23728bc808a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:48.049378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255780f-9c57-4be9-b1dc-a23728bc808a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:48.055361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c9c2c-660d-4579-8dae-7259fe51a83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:48.056083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c9c2c-660d-4579-8dae-7259fe51a83d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:48.056475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c9c2c-660d-4579-8dae-7259fe51a83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:48.057036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c9c2c-660d-4579-8dae-7259fe51a83d admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:48.057691 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c9c2c-660d-4579-8dae-7259fe51a83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:48.059588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c9c2c-660d-4579-8dae-7259fe51a83d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:48.079801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1049/2079] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:48.083439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c9c2c-660d-4579-8dae-7259fe51a83d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:48.101767 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c9c2c-660d-4579-8dae-7259fe51a83d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:48.110992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64164639-ccf5-4632-8262-31a0cfec9d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:48.112110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64164639-ccf5-4632-8262-31a0cfec9d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:48.113128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64164639-ccf5-4632-8262-31a0cfec9d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:48.113412 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64164639-ccf5-4632-8262-31a0cfec9d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:48.114128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64164639-ccf5-4632-8262-31a0cfec9d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:48.117166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64164639-ccf5-4632-8262-31a0cfec9d4d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:48.135271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1050/2080] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:48.136416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64164639-ccf5-4632-8262-31a0cfec9d4d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:48.146834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:48.153995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64164639-ccf5-4632-8262-31a0cfec9d4d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:48.159810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2eb88a-cc3e-4a20-8b97-920436512937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:48.160725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eb88a-cc3e-4a20-8b97-920436512937 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:48.161245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eb88a-cc3e-4a20-8b97-920436512937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:48.161711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eb88a-cc3e-4a20-8b97-920436512937 admin admin] PATH_INFO: `/v3/projects/8191457181714862bbeb70259fa86971/users/81e89f8fb70d49d18cd0c81c98dd4add/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:48.180118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e4dc9569-517c-4d10-93da-a750ca1c5854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0648d7f8df634273bad28a7058d6bdba'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:48.202112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eb88a-cc3e-4a20-8b97-920436512937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8191457181714862bbeb70259fa86971, user_id=81e89f8fb70d49d18cd0c81c98dd4add, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:48.207625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eb88a-cc3e-4a20-8b97-920436512937 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:48.235956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e4dc9569-517c-4d10-93da-a750ca1c5854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0648d7f8df634273bad28a7058d6bdba'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:48.251164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e4dc9569-517c-4d10-93da-a750ca1c5854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0648d7f8df634273bad28a7058d6bdba'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:48.311084 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1051/2081] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:48 2022] PUT /identity/v3/projects/8191457181714862bbeb70259fa86971/users/81e89f8fb70d49d18cd0c81c98dd4add/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:48.316396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bf90b-9075-4faf-85ec-a8051339cfed None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:48.316743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bf90b-9075-4faf-85ec-a8051339cfed None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:48.317272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bf90b-9075-4faf-85ec-a8051339cfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:48.344932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-677bf90b-9075-4faf-85ec-a8051339cfed None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:48.346637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-677bf90b-9075-4faf-85ec-a8051339cfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:48.365637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e4dc9569-517c-4d10-93da-a750ca1c5854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0648d7f8df634273bad28a7058d6bdba'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:48.369855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e4dc9569-517c-4d10-93da-a750ca1c5854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0648d7f8df634273bad28a7058d6bdba'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:48.378560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1031/2082] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:47 2022] DELETE /identity/v3/users/0648d7f8df634273bad28a7058d6bdba => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:48.383372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d806-7ab7-4961-a824-8fd9b0ca8028 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:48.384411 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d806-7ab7-4961-a824-8fd9b0ca8028 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:48.384908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d806-7ab7-4961-a824-8fd9b0ca8028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:48.422128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-25a9d806-7ab7-4961-a824-8fd9b0ca8028 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:48.425218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-25a9d806-7ab7-4961-a824-8fd9b0ca8028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:48.432954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1052/2083] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:48.443746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bf90b-9075-4faf-85ec-a8051339cfed None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:48.445593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-677bf90b-9075-4faf-85ec-a8051339cfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:48.520803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bf90b-9075-4faf-85ec-a8051339cfed None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:48.564556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1032/2084] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:48.574491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d806-7ab7-4961-a824-8fd9b0ca8028 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:48.586441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4526b29-2cfb-4d87-946b-97725973fb1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:48.587453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4526b29-2cfb-4d87-946b-97725973fb1f service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:48.587953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4526b29-2cfb-4d87-946b-97725973fb1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:48.588554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4526b29-2cfb-4d87-946b-97725973fb1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:48.607494 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4526b29-2cfb-4d87-946b-97725973fb1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:48.608944 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4526b29-2cfb-4d87-946b-97725973fb1f service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:48.622012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:48.627708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1053/2085] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:48.658872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d806-7ab7-4961-a824-8fd9b0ca8028 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:48.664136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d491b-b68d-426b-90d1-0042e583526b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:48.664872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d491b-b68d-426b-90d1-0042e583526b service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:48.665154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d491b-b68d-426b-90d1-0042e583526b service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:48.665428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d491b-b68d-426b-90d1-0042e583526b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:48.676770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d491b-b68d-426b-90d1-0042e583526b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:48.679226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d491b-b68d-426b-90d1-0042e583526b service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:48.691465 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1033/2086] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:49.410751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4526b29-2cfb-4d87-946b-97725973fb1f service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:49.412558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b4526b29-2cfb-4d87-946b-97725973fb1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:49.444830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4526b29-2cfb-4d87-946b-97725973fb1f service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:49.459275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:49.462019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:49.462766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:49.463052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] PATH_INFO: `/v3/projects/64e6eebfe897446580cc756a85558f67` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:49.465920 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e6eebfe897446580cc756a85558f67)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:49.469790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:49.666656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e6eebfe897446580cc756a85558f67', 'request_id': 'req-54637f94-5ea7-44a0-a237-edf00455e5d1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:49.670757 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e6eebfe897446580cc756a85558f67', 'request_id': 'req-54637f94-5ea7-44a0-a237-edf00455e5d1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:49.677972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e6eebfe897446580cc756a85558f67', 'request_id': 'req-54637f94-5ea7-44a0-a237-edf00455e5d1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:49.732059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] The token cache is being invalidate because project 64e6eebfe897446580cc756a85558f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:49.732686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:49.734416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-54637f94-5ea7-44a0-a237-edf00455e5d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:49.741722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1054/2087] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:49 2022] DELETE /identity/v3/projects/64e6eebfe897446580cc756a85558f67 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:49.745369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d491b-b68d-426b-90d1-0042e583526b service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:49.757113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d491b-b68d-426b-90d1-0042e583526b service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:49.827893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87cec4-1f25-4267-bf57-4a9991ee6b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:49.829211 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87cec4-1f25-4267-bf57-4a9991ee6b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:49.829707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87cec4-1f25-4267-bf57-4a9991ee6b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:49.831669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87cec4-1f25-4267-bf57-4a9991ee6b3f admin admin] PATH_INFO: `/v3/users/a2b2c597a5654ad19b43e3de3f1dd56d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:49.834019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87cec4-1f25-4267-bf57-4a9991ee6b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b2c597a5654ad19b43e3de3f1dd56d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:49.836963 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87cec4-1f25-4267-bf57-4a9991ee6b3f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:50.124922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6e87cec4-1f25-4267-bf57-4a9991ee6b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b2c597a5654ad19b43e3de3f1dd56d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:50.138400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6e87cec4-1f25-4267-bf57-4a9991ee6b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b2c597a5654ad19b43e3de3f1dd56d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:50.146862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6e87cec4-1f25-4267-bf57-4a9991ee6b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b2c597a5654ad19b43e3de3f1dd56d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:50.233193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6e87cec4-1f25-4267-bf57-4a9991ee6b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b2c597a5654ad19b43e3de3f1dd56d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:50.235254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6e87cec4-1f25-4267-bf57-4a9991ee6b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b2c597a5654ad19b43e3de3f1dd56d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:50.261889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1034/2088] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:49 2022] DELETE /identity/v3/users/a2b2c597a5654ad19b43e3de3f1dd56d => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:50.343227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:50.486929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa04f3c-ff94-4769-aab7-3c5d9e228a2c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:50.487790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa04f3c-ff94-4769-aab7-3c5d9e228a2c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:50.488341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa04f3c-ff94-4769-aab7-3c5d9e228a2c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:50.491959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1055/2089] 10.222.0.59 () {54 vars in 921 bytes} [Fri Sep 16 08:06:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 08:06:50.509909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5e7d7-dd7a-445e-8e09-9de0f94cd9e3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:50.512587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5e7d7-dd7a-445e-8e09-9de0f94cd9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:50.513985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5e7d7-dd7a-445e-8e09-9de0f94cd9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:50.537651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-06f5e7d7-dd7a-445e-8e09-9de0f94cd9e3 None None] MFA Rules not processed for user `c2dca05dfe8b4af0a02ea93fbe29c556`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:50.538935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-06f5e7d7-dd7a-445e-8e09-9de0f94cd9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:50.621174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1035/2090] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Sep 16 08:06:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:50.632962 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa04f3c-ff94-4769-aab7-3c5d9e228a2c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:50.636239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9fa04f3c-ff94-4769-aab7-3c5d9e228a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:50.655882 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa04f3c-ff94-4769-aab7-3c5d9e228a2c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:50.667175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a08ee6-907e-4055-8a04-a150b377aefc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:50.668458 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a08ee6-907e-4055-8a04-a150b377aefc service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:50.669289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a08ee6-907e-4055-8a04-a150b377aefc service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:50.669686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a08ee6-907e-4055-8a04-a150b377aefc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:50.690533 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a08ee6-907e-4055-8a04-a150b377aefc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:50.692933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a08ee6-907e-4055-8a04-a150b377aefc service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:50.706328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1056/2091] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:50.742395 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5e7d7-dd7a-445e-8e09-9de0f94cd9e3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:50.746201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-06f5e7d7-dd7a-445e-8e09-9de0f94cd9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:50.918118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5e7d7-dd7a-445e-8e09-9de0f94cd9e3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:50.933213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9caf04-5a0a-449e-a955-5054fe728986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:50.934785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9caf04-5a0a-449e-a955-5054fe728986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:50.936674 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9caf04-5a0a-449e-a955-5054fe728986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:50.937122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9caf04-5a0a-449e-a955-5054fe728986 admin admin] PATH_INFO: `/v3/users/2757da8a75de4210bbb945fa12335567` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:50.953659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9caf04-5a0a-449e-a955-5054fe728986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2757da8a75de4210bbb945fa12335567)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:50.959110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9caf04-5a0a-449e-a955-5054fe728986 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:51.086714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:51.238834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a08ee6-907e-4055-8a04-a150b377aefc service placement] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:51.240685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-26a08ee6-907e-4055-8a04-a150b377aefc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:51.262177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9d9caf04-5a0a-449e-a955-5054fe728986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2757da8a75de4210bbb945fa12335567'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:51.265923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a08ee6-907e-4055-8a04-a150b377aefc service placement] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:51.277748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:51.280217 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:51.288898 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9d9caf04-5a0a-449e-a955-5054fe728986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2757da8a75de4210bbb945fa12335567'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:51.297576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:51.298064 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] PATH_INFO: `/v3/projects/d46efe8c9af2406e9fcee58c80dfa2c1` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:51.306216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9d9caf04-5a0a-449e-a955-5054fe728986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2757da8a75de4210bbb945fa12335567'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:51.306472 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d46efe8c9af2406e9fcee58c80dfa2c1)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:51.310979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:51.419045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9d9caf04-5a0a-449e-a955-5054fe728986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2757da8a75de4210bbb945fa12335567'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:51.419723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9d9caf04-5a0a-449e-a955-5054fe728986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2757da8a75de4210bbb945fa12335567'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:51.428652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1036/2092] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:50 2022] DELETE /identity/v3/users/2757da8a75de4210bbb945fa12335567 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:51.483946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd46efe8c9af2406e9fcee58c80dfa2c1', 'request_id': 'req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:51.500575 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd46efe8c9af2406e9fcee58c80dfa2c1', 'request_id': 'req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:51.510888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd46efe8c9af2406e9fcee58c80dfa2c1', 'request_id': 'req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:51.589817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] The token cache is being invalidate because project d46efe8c9af2406e9fcee58c80dfa2c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:51.590382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:51.591485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:51.595568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1057/2093] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:51 2022] DELETE /identity/v3/projects/d46efe8c9af2406e9fcee58c80dfa2c1 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:51.600179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110c4e3-70ed-4a64-b3ac-2e1b2fd70e0f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:51.600463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110c4e3-70ed-4a64-b3ac-2e1b2fd70e0f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:51.600744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110c4e3-70ed-4a64-b3ac-2e1b2fd70e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:51.647996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7110c4e3-70ed-4a64-b3ac-2e1b2fd70e0f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:51.649335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7110c4e3-70ed-4a64-b3ac-2e1b2fd70e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:51.769690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:51.770865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1037/2094] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:51.771321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:51.773097 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8d8bc-b29b-4939-abf3-02982cc159ba None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:51.773619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8d8bc-b29b-4939-abf3-02982cc159ba None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:51.774146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8d8bc-b29b-4939-abf3-02982cc159ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:51.788956 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1c17-61b3-4b08-8725-ce8d09d1ca7e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:51.795089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:51.795954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:51.802778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:51.803395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] PATH_INFO: `/v3/projects/8b54e1e0edf74e53b77cce236388b9cb` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:51.805852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b54e1e0edf74e53b77cce236388b9cb)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:51.810968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:51.811940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-13a8d8bc-b29b-4939-abf3-02982cc159ba None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:51.814559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-13a8d8bc-b29b-4939-abf3-02982cc159ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:51.856180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1038/2095] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:51.857940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ebf90-b14f-4080-8bcb-71c227958204 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:51.858221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ebf90-b14f-4080-8bcb-71c227958204 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:51.863322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ebf90-b14f-4080-8bcb-71c227958204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:51.891455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b70ebf90-b14f-4080-8bcb-71c227958204 None None] MFA Rules not processed for user `66702831c1ca4158976920f6be2cc80c`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:51.892292 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b70ebf90-b14f-4080-8bcb-71c227958204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:51.967881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b54e1e0edf74e53b77cce236388b9cb', 'request_id': 'req-45497925-5336-4cb6-b3ba-ce059ffd9038'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:51.985508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b54e1e0edf74e53b77cce236388b9cb', 'request_id': 'req-45497925-5336-4cb6-b3ba-ce059ffd9038'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:52.001929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b54e1e0edf74e53b77cce236388b9cb', 'request_id': 'req-45497925-5336-4cb6-b3ba-ce059ffd9038'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:52.008271 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1039/2096] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:52.009786 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ebf90-b14f-4080-8bcb-71c227958204 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:52.024624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ebf90-b14f-4080-8bcb-71c227958204 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:52.111550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cca5a8-a1db-40b8-ac5d-155337ad482a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:52.114921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cca5a8-a1db-40b8-ac5d-155337ad482a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:52.114921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cca5a8-a1db-40b8-ac5d-155337ad482a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:52.115171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cca5a8-a1db-40b8-ac5d-155337ad482a admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:52.115931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cca5a8-a1db-40b8-ac5d-155337ad482a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:52.116163 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cca5a8-a1db-40b8-ac5d-155337ad482a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:52.119036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cca5a8-a1db-40b8-ac5d-155337ad482a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:52.132864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1040/2097] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:06:52.133717 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cca5a8-a1db-40b8-ac5d-155337ad482a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:52.135775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c8cca5a8-a1db-40b8-ac5d-155337ad482a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:52.155074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] The token cache is being invalidate because project 8b54e1e0edf74e53b77cce236388b9cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:52.155074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:52.155074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:52.155074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1058/2098] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:51 2022] DELETE /identity/v3/projects/8b54e1e0edf74e53b77cce236388b9cb => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:06:52.155423 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cca5a8-a1db-40b8-ac5d-155337ad482a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:52.155514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:52.155514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:52.163165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526f480-0772-4238-a856-651672beb928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:52.163472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526f480-0772-4238-a856-651672beb928 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:52.163768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526f480-0772-4238-a856-651672beb928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:52.164120 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526f480-0772-4238-a856-651672beb928 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:52.164844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526f480-0772-4238-a856-651672beb928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:52.166938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526f480-0772-4238-a856-651672beb928 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:52.226049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45497925-5336-4cb6-b3ba-ce059ffd9038 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:52.233494 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023b627-7d67-4e0c-a884-b40490fcf5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:52.234120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023b627-7d67-4e0c-a884-b40490fcf5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:52.234423 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023b627-7d67-4e0c-a884-b40490fcf5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:52.234780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023b627-7d67-4e0c-a884-b40490fcf5b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:52.235365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023b627-7d67-4e0c-a884-b40490fcf5b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:52.237342 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023b627-7d67-4e0c-a884-b40490fcf5b0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:52.280835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1041/2099] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:52 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:52.281695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526f480-0772-4238-a856-651672beb928 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:52.285719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5526f480-0772-4238-a856-651672beb928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:52.336856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1059/2100] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:52 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:52.337465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023b627-7d67-4e0c-a884-b40490fcf5b0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:52.340692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a023b627-7d67-4e0c-a884-b40490fcf5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:52.420219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526f480-0772-4238-a856-651672beb928 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:52.463804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023b627-7d67-4e0c-a884-b40490fcf5b0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:52.471663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc66447-6b07-4946-b675-5e2171caeb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:52.472403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc66447-6b07-4946-b675-5e2171caeb04 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:52.472985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc66447-6b07-4946-b675-5e2171caeb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:52.473894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc66447-6b07-4946-b675-5e2171caeb04 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:52.474924 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc66447-6b07-4946-b675-5e2171caeb04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:52.477302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc66447-6b07-4946-b675-5e2171caeb04 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:52.503937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:52.504830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:52.505535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:52.505535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] PATH_INFO: `/v3/users/3d9881a127cb45fca47494914985a44f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:52.508407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d9881a127cb45fca47494914985a44f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:52.510984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:52.538325 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:52.574732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1060/2101] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:52 2022] POST /identity/v3/users => generated 299 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:52.574732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc66447-6b07-4946-b675-5e2171caeb04 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:52.596433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc66447-6b07-4946-b675-5e2171caeb04 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:52.607256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fe36e-194b-42b0-8b5f-389223d16f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:52.608155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fe36e-194b-42b0-8b5f-389223d16f32 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:52.608631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fe36e-194b-42b0-8b5f-389223d16f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:52.609097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fe36e-194b-42b0-8b5f-389223d16f32 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:52.610021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fe36e-194b-42b0-8b5f-389223d16f32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:52.612413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fe36e-194b-42b0-8b5f-389223d16f32 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:52.749360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1061/2102] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:52 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:52.750550 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fe36e-194b-42b0-8b5f-389223d16f32 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:52.762038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d9881a127cb45fca47494914985a44f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:52.764309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fe36e-194b-42b0-8b5f-389223d16f32 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:52.774735 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab678455-ff47-44b2-855a-10f1522b0e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:52.775778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab678455-ff47-44b2-855a-10f1522b0e01 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:52.775778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab678455-ff47-44b2-855a-10f1522b0e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:52.775979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab678455-ff47-44b2-855a-10f1522b0e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:52.776524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab678455-ff47-44b2-855a-10f1522b0e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:52.778974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d9881a127cb45fca47494914985a44f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:52.779480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab678455-ff47-44b2-855a-10f1522b0e01 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:52.797966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1062/2103] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:52.802343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab678455-ff47-44b2-855a-10f1522b0e01 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:52.803363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d9881a127cb45fca47494914985a44f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:52.804455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ab678455-ff47-44b2-855a-10f1522b0e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:52.824145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=78,keystone:DELETE=18,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:52.824884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab678455-ff47-44b2-855a-10f1522b0e01 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:52.830456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:52.831407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:52.831829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:52.832250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] PATH_INFO: `/v3/users/7ae69073291a4d279d213cd5d3147236` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:52.834439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae69073291a4d279d213cd5d3147236)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:52.837419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:52.872894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d9881a127cb45fca47494914985a44f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:52.876428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d9881a127cb45fca47494914985a44f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:52.883348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1042/2104] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:52 2022] DELETE /identity/v3/users/3d9881a127cb45fca47494914985a44f => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:52.886372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:52.887005 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:52.995753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd786f-5ccb-4ed1-bba3-9825913775b1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:53.002159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd93ae-c822-46f2-a0f3-3b5b4ebb16de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:53.002830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd93ae-c822-46f2-a0f3-3b5b4ebb16de admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:53.003432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd93ae-c822-46f2-a0f3-3b5b4ebb16de admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:53.004151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd93ae-c822-46f2-a0f3-3b5b4ebb16de admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:53.005341 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd93ae-c822-46f2-a0f3-3b5b4ebb16de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:53.006997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd93ae-c822-46f2-a0f3-3b5b4ebb16de admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:53.018492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1043/2105] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:53.019465 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd93ae-c822-46f2-a0f3-3b5b4ebb16de admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:53.022547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-facd93ae-c822-46f2-a0f3-3b5b4ebb16de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:53.036477 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd93ae-c822-46f2-a0f3-3b5b4ebb16de admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:53.042637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d0419-1005-4aeb-a54c-827bf4d3d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:53.042637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d0419-1005-4aeb-a54c-827bf4d3d384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:53.043076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d0419-1005-4aeb-a54c-827bf4d3d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:53.043283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d0419-1005-4aeb-a54c-827bf4d3d384 admin admin] PATH_INFO: `/v3/projects/d5e9862850ba48d286fe4c4e52be7891/users/e4107a0136834061828fb975136de88e/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:53.075269 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d0419-1005-4aeb-a54c-827bf4d3d384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e9862850ba48d286fe4c4e52be7891, user_id=e4107a0136834061828fb975136de88e, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:53.080802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d0419-1005-4aeb-a54c-827bf4d3d384 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:53.158129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1044/2106] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:53 2022] PUT /identity/v3/projects/d5e9862850ba48d286fe4c4e52be7891/users/e4107a0136834061828fb975136de88e/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:53.159419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d0419-1005-4aeb-a54c-827bf4d3d384 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:53.161267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae69073291a4d279d213cd5d3147236'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:53.206185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae69073291a4d279d213cd5d3147236'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:53.223644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae69073291a4d279d213cd5d3147236'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:53.292154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d0419-1005-4aeb-a54c-827bf4d3d384 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:53.295203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae69073291a4d279d213cd5d3147236'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:53.296994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae69073291a4d279d213cd5d3147236'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:53.302640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc494d4-2f9e-4f17-b7f0-8d3acd52b6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:53.303720 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc494d4-2f9e-4f17-b7f0-8d3acd52b6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:53.304191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc494d4-2f9e-4f17-b7f0-8d3acd52b6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:53.304792 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc494d4-2f9e-4f17-b7f0-8d3acd52b6f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:53.306441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc494d4-2f9e-4f17-b7f0-8d3acd52b6f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:53.309455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc494d4-2f9e-4f17-b7f0-8d3acd52b6f1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:53.311180 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1063/2107] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:06:52 2022] DELETE /identity/v3/users/7ae69073291a4d279d213cd5d3147236 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:53.312113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:53.314099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:53.329020 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1045/2108] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:53.331128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f7269-adfa-448a-8420-0e3fd43c6c5b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:53.331727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f7269-adfa-448a-8420-0e3fd43c6c5b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:53.332016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f7269-adfa-448a-8420-0e3fd43c6c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:53.367480 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a98f7269-adfa-448a-8420-0e3fd43c6c5b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:53.368743 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a98f7269-adfa-448a-8420-0e3fd43c6c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:53.432847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d2722-5774-4043-8149-1c83f096e0c4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:53.450231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961964c-b554-4f9a-bcc0-9e55847a7850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:53.450883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961964c-b554-4f9a-bcc0-9e55847a7850 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:53.451170 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961964c-b554-4f9a-bcc0-9e55847a7850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:53.451445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961964c-b554-4f9a-bcc0-9e55847a7850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:53.452097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961964c-b554-4f9a-bcc0-9e55847a7850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:53.456778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961964c-b554-4f9a-bcc0-9e55847a7850 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:53.484377 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1064/2109] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:53.485420 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961964c-b554-4f9a-bcc0-9e55847a7850 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:53.497745 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7961964c-b554-4f9a-bcc0-9e55847a7850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:53.511663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961964c-b554-4f9a-bcc0-9e55847a7850 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:53.517885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3c10-a90d-4e2a-8b3e-50f525530e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:53.519597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3c10-a90d-4e2a-8b3e-50f525530e53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:53.519878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3c10-a90d-4e2a-8b3e-50f525530e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:53.520381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3c10-a90d-4e2a-8b3e-50f525530e53 admin admin] PATH_INFO: `/v3/projects/5002673965844cb887af41eccd644e8c/users/f62ac79d3db74b7fb0dbc901a95c9e73/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:53.538790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d3c10-a90d-4e2a-8b3e-50f525530e53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5002673965844cb887af41eccd644e8c, user_id=f62ac79d3db74b7fb0dbc901a95c9e73, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:53.542025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d3c10-a90d-4e2a-8b3e-50f525530e53 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:53.557427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1046/2110] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:53.558535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f7269-adfa-448a-8420-0e3fd43c6c5b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:53.574964 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f7269-adfa-448a-8420-0e3fd43c6c5b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:53.580742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236951f-6155-4408-98cb-839ff00d5247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:53.581801 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236951f-6155-4408-98cb-839ff00d5247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:53.582126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236951f-6155-4408-98cb-839ff00d5247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:53.582408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236951f-6155-4408-98cb-839ff00d5247 admin admin] PATH_INFO: `/v3/projects/d5e9862850ba48d286fe4c4e52be7891/users/e4107a0136834061828fb975136de88e/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:53.584512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236951f-6155-4408-98cb-839ff00d5247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e9862850ba48d286fe4c4e52be7891, user_id=e4107a0136834061828fb975136de88e, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:53.588460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236951f-6155-4408-98cb-839ff00d5247 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:53.603998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1047/2111] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:53 2022] PUT /identity/v3/projects/d5e9862850ba48d286fe4c4e52be7891/users/e4107a0136834061828fb975136de88e/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:53.604944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236951f-6155-4408-98cb-839ff00d5247 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:53.609864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7236951f-6155-4408-98cb-839ff00d5247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:53.627555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1065/2112] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:53 2022] PUT /identity/v3/projects/5002673965844cb887af41eccd644e8c/users/f62ac79d3db74b7fb0dbc901a95c9e73/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:53.631045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3c10-a90d-4e2a-8b3e-50f525530e53 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:53.634547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b06d3c10-a90d-4e2a-8b3e-50f525530e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:53.755773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236951f-6155-4408-98cb-839ff00d5247 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:53.797802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3c10-a90d-4e2a-8b3e-50f525530e53 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:53.809478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8514fed-5b8c-4f88-bd12-1f2109c862fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:53.810313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8514fed-5b8c-4f88-bd12-1f2109c862fc service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:53.810584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8514fed-5b8c-4f88-bd12-1f2109c862fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:53.813802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8514fed-5b8c-4f88-bd12-1f2109c862fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:53.875550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:53.876238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:53.877936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:53.878239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] PATH_INFO: `/v3/projects/13c8229ddcc04e29825df27ad96ad7f1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:53.879495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=13c8229ddcc04e29825df27ad96ad7f1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:53.881705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:53.946130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8514fed-5b8c-4f88-bd12-1f2109c862fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:53.949653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8514fed-5b8c-4f88-bd12-1f2109c862fc service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:53.991327 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1066/2113] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:53.993315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eecf87-0c1e-4003-86de-b3ddabd9216a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:53.993607 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eecf87-0c1e-4003-86de-b3ddabd9216a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:53.994505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eecf87-0c1e-4003-86de-b3ddabd9216a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:54.004501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13c8229ddcc04e29825df27ad96ad7f1', 'request_id': 'req-da29f484-20e2-4014-8518-15cfa77fae5c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:54.015453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13c8229ddcc04e29825df27ad96ad7f1', 'request_id': 'req-da29f484-20e2-4014-8518-15cfa77fae5c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:54.030036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13c8229ddcc04e29825df27ad96ad7f1', 'request_id': 'req-da29f484-20e2-4014-8518-15cfa77fae5c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:54.051774 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-26eecf87-0c1e-4003-86de-b3ddabd9216a None None] MFA Rules not processed for user `f62ac79d3db74b7fb0dbc901a95c9e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:54.053115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-26eecf87-0c1e-4003-86de-b3ddabd9216a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:54.159408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] The token cache is being invalidate because project 13c8229ddcc04e29825df27ad96ad7f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:54.159743 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:54.162764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:54.168009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1067/2115] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:54.171169 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4a9ae-4e8c-41de-a49d-c6e4b3f108a4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:54.171736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4a9ae-4e8c-41de-a49d-c6e4b3f108a4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:54.172029 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4a9ae-4e8c-41de-a49d-c6e4b3f108a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:54.172464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1048/2115] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:53 2022] DELETE /identity/v3/projects/13c8229ddcc04e29825df27ad96ad7f1 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:54.175375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:54.176996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:54.216455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7bd4a9ae-4e8c-41de-a49d-c6e4b3f108a4 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:54.219124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7bd4a9ae-4e8c-41de-a49d-c6e4b3f108a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:54.322332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29f484-20e2-4014-8518-15cfa77fae5c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:54.329866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c0e84-9678-45bd-bb0a-6a59ca4774bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:54.330800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0e84-9678-45bd-bb0a-6a59ca4774bd admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:54.331335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0e84-9678-45bd-bb0a-6a59ca4774bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:54.332307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0e84-9678-45bd-bb0a-6a59ca4774bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:54.334078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c0e84-9678-45bd-bb0a-6a59ca4774bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:54.336996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c0e84-9678-45bd-bb0a-6a59ca4774bd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:54.369934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1068/2116] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:54.380607 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb73e8-b05b-4896-9700-b8b0b93a9487 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:54.381251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb73e8-b05b-4896-9700-b8b0b93a9487 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:54.381251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb73e8-b05b-4896-9700-b8b0b93a9487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:54.422056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-95bb73e8-b05b-4896-9700-b8b0b93a9487 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:54.423362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-95bb73e8-b05b-4896-9700-b8b0b93a9487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:54.436904 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1069/2117] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:54.438249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb73e8-b05b-4896-9700-b8b0b93a9487 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:54.451607 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb73e8-b05b-4896-9700-b8b0b93a9487 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:54.457114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3d590-569b-458b-922d-09ea275e39a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:54.458491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3d590-569b-458b-922d-09ea275e39a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:54.458823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3d590-569b-458b-922d-09ea275e39a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:54.459176 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3d590-569b-458b-922d-09ea275e39a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:54.459895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3d590-569b-458b-922d-09ea275e39a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:54.460203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3d590-569b-458b-922d-09ea275e39a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:54.461955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3d590-569b-458b-922d-09ea275e39a5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:54.483466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1049/2118] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:54 2022] POST /identity/v3/projects => generated 377 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:54.484564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c0e84-9678-45bd-bb0a-6a59ca4774bd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:54.495685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1070/2119] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:06:54.497370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3d590-569b-458b-922d-09ea275e39a5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:54.521937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3d590-569b-458b-922d-09ea275e39a5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:54.623296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c0e84-9678-45bd-bb0a-6a59ca4774bd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:54.634764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301f884-2801-4e05-ba79-0ad40f73e5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:54.635361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301f884-2801-4e05-ba79-0ad40f73e5cc admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:54.635644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301f884-2801-4e05-ba79-0ad40f73e5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:54.635921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301f884-2801-4e05-ba79-0ad40f73e5cc admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:54.637179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301f884-2801-4e05-ba79-0ad40f73e5cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:54.639298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301f884-2801-4e05-ba79-0ad40f73e5cc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:54.657920 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea813ee3-a91d-4971-aceb-f711381e0b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:54.659600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea813ee3-a91d-4971-aceb-f711381e0b48 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:54.660027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea813ee3-a91d-4971-aceb-f711381e0b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:54.660619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea813ee3-a91d-4971-aceb-f711381e0b48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:54.662828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea813ee3-a91d-4971-aceb-f711381e0b48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:54.665168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea813ee3-a91d-4971-aceb-f711381e0b48 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:54.753884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1050/2120] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:54 2022] POST /identity/v3/users => generated 295 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:54.765055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301f884-2801-4e05-ba79-0ad40f73e5cc admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:54.792071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301f884-2801-4e05-ba79-0ad40f73e5cc admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:54.811686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448b9b4-c851-4d31-a5b2-abe7c69785c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:54.812585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448b9b4-c851-4d31-a5b2-abe7c69785c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:54.813008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448b9b4-c851-4d31-a5b2-abe7c69785c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:54.813429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448b9b4-c851-4d31-a5b2-abe7c69785c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:54.815004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448b9b4-c851-4d31-a5b2-abe7c69785c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:54.817159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448b9b4-c851-4d31-a5b2-abe7c69785c7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:54.817310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1071/2121] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:54 2022] POST /identity/v3/projects => generated 383 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:54.823655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea813ee3-a91d-4971-aceb-f711381e0b48 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:54.835422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:54.847070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1051/2122] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:54.850744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448b9b4-c851-4d31-a5b2-abe7c69785c7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:54.867716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448b9b4-c851-4d31-a5b2-abe7c69785c7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:54.994920 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea813ee3-a91d-4971-aceb-f711381e0b48 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:55.005186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcc349-a11f-4557-9081-c7c27ff78d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:55.006300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcc349-a11f-4557-9081-c7c27ff78d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:55.006584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcc349-a11f-4557-9081-c7c27ff78d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:55.006886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcc349-a11f-4557-9081-c7c27ff78d05 admin admin] PATH_INFO: `/v3/projects/f501df73194b47b3b5479f651eee3832/users/63748e65c9084f96ad42664be2ea936f/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:55.009511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65340e95-7e6a-45cf-b17d-1f83273f983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:55.010421 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65340e95-7e6a-45cf-b17d-1f83273f983d admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:55.010739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65340e95-7e6a-45cf-b17d-1f83273f983d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:55.010994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65340e95-7e6a-45cf-b17d-1f83273f983d admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:55.014165 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65340e95-7e6a-45cf-b17d-1f83273f983d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:55.018186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65340e95-7e6a-45cf-b17d-1f83273f983d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:55.058238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcc349-a11f-4557-9081-c7c27ff78d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f501df73194b47b3b5479f651eee3832, user_id=63748e65c9084f96ad42664be2ea936f, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:55.062075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcc349-a11f-4557-9081-c7c27ff78d05 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:55.165299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:55.179947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1052/2123] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:54 2022] PUT /identity/v3/projects/f501df73194b47b3b5479f651eee3832/users/63748e65c9084f96ad42664be2ea936f/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:55.183544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1072/2124] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:54 2022] POST /identity/v3/users => generated 292 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:55.184781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcc349-a11f-4557-9081-c7c27ff78d05 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:55.190078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65340e95-7e6a-45cf-b17d-1f83273f983d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:55.242264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65340e95-7e6a-45cf-b17d-1f83273f983d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:55.323882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcc349-a11f-4557-9081-c7c27ff78d05 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:55.330094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb3b7f-8a43-4c12-81e4-4a1287e29a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:55.331032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb3b7f-8a43-4c12-81e4-4a1287e29a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:55.331729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb3b7f-8a43-4c12-81e4-4a1287e29a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:55.332181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb3b7f-8a43-4c12-81e4-4a1287e29a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:55.333176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb3b7f-8a43-4c12-81e4-4a1287e29a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:55.335238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb3b7f-8a43-4c12-81e4-4a1287e29a52 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:55.337511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7f323-edf8-4c02-9303-1c5041cf383e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:55.338566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7f323-edf8-4c02-9303-1c5041cf383e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:55.339014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7f323-edf8-4c02-9303-1c5041cf383e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:55.339681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7f323-edf8-4c02-9303-1c5041cf383e admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:55.341571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7f323-edf8-4c02-9303-1c5041cf383e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:55.345632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7f323-edf8-4c02-9303-1c5041cf383e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:55.355212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1053/2125] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:55.358747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1073/2126] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:55.359995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7f323-edf8-4c02-9303-1c5041cf383e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:55.364998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb3b7f-8a43-4c12-81e4-4a1287e29a52 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:55.367642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-73fb3b7f-8a43-4c12-81e4-4a1287e29a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:55.374751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7f323-edf8-4c02-9303-1c5041cf383e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:55.380384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc101-5f05-4f45-91d9-4b2ccf5fed00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:55.381201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc101-5f05-4f45-91d9-4b2ccf5fed00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:55.381703 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc101-5f05-4f45-91d9-4b2ccf5fed00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:55.382182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc101-5f05-4f45-91d9-4b2ccf5fed00 admin admin] PATH_INFO: `/v3/projects/f501df73194b47b3b5479f651eee3832/users/63748e65c9084f96ad42664be2ea936f/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:55.385079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bc101-5f05-4f45-91d9-4b2ccf5fed00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f501df73194b47b3b5479f651eee3832, user_id=63748e65c9084f96ad42664be2ea936f, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:55.387663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bc101-5f05-4f45-91d9-4b2ccf5fed00 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:55.401171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1074/2127] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:55 2022] PUT /identity/v3/projects/f501df73194b47b3b5479f651eee3832/users/63748e65c9084f96ad42664be2ea936f/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:55.402445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc101-5f05-4f45-91d9-4b2ccf5fed00 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:55.430087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb3b7f-8a43-4c12-81e4-4a1287e29a52 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:55.519268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc101-5f05-4f45-91d9-4b2ccf5fed00 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:55.525103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812682e-6656-45fe-ab74-a40db4292ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:55.525907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812682e-6656-45fe-ab74-a40db4292ffb admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:55.526257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812682e-6656-45fe-ab74-a40db4292ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:55.526461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812682e-6656-45fe-ab74-a40db4292ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:55.527138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812682e-6656-45fe-ab74-a40db4292ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:55.528816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812682e-6656-45fe-ab74-a40db4292ffb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:55.531102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:55.532038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:55.532530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:55.533017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] PATH_INFO: `/v3/projects/90cd8bc81fff4256877db5ac6b9df4ea` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:55.535341 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90cd8bc81fff4256877db5ac6b9df4ea)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:55.538494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:55.556673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1075/2128] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:55.556673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ce815-f2ef-4173-a203-88ba31bc8125 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:55.556673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ce815-f2ef-4173-a203-88ba31bc8125 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:55.556673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ce815-f2ef-4173-a203-88ba31bc8125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:55.611821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-416ce815-f2ef-4173-a203-88ba31bc8125 None None] MFA Rules not processed for user `63748e65c9084f96ad42664be2ea936f`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:55.613566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-416ce815-f2ef-4173-a203-88ba31bc8125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:55.706773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1076/2129] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:55.708613 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ce815-f2ef-4173-a203-88ba31bc8125 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:55.731664 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ce815-f2ef-4173-a203-88ba31bc8125 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:55.748042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90cd8bc81fff4256877db5ac6b9df4ea', 'request_id': 'req-78b425ee-1fed-4b7a-a542-e31e531fe840'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:55.761853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90cd8bc81fff4256877db5ac6b9df4ea', 'request_id': 'req-78b425ee-1fed-4b7a-a542-e31e531fe840'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:55.773361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90cd8bc81fff4256877db5ac6b9df4ea', 'request_id': 'req-78b425ee-1fed-4b7a-a542-e31e531fe840'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:55.831964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ef2e2-f8e5-4e68-8a4b-5d6a608884bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:55.833171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ef2e2-f8e5-4e68-8a4b-5d6a608884bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:55.833454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ef2e2-f8e5-4e68-8a4b-5d6a608884bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:55.833729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ef2e2-f8e5-4e68-8a4b-5d6a608884bd admin admin] PATH_INFO: `/v3/projects/9bf83d3dec5849619a9b687124cd11ac/users/fa397bd1eff7495387deaf83428270b8/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:55.853988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ef2e2-f8e5-4e68-8a4b-5d6a608884bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf83d3dec5849619a9b687124cd11ac, user_id=fa397bd1eff7495387deaf83428270b8, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:55.857389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ef2e2-f8e5-4e68-8a4b-5d6a608884bd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:55.898868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] The token cache is being invalidate because project 90cd8bc81fff4256877db5ac6b9df4ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:06:55.899375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:55.900254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:06:55.901740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1054/2130] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:06:55 2022] DELETE /identity/v3/projects/90cd8bc81fff4256877db5ac6b9df4ea => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:55.908723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:55.911841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:55.965568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1077/2131] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:55 2022] PUT /identity/v3/projects/9bf83d3dec5849619a9b687124cd11ac/users/fa397bd1eff7495387deaf83428270b8/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:55.967816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140f17e-45ef-4daf-8cd8-9292e756f50c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:55.968348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140f17e-45ef-4daf-8cd8-9292e756f50c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:55.971407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140f17e-45ef-4daf-8cd8-9292e756f50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.007480 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b425ee-1fed-4b7a-a542-e31e531fe840 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.025179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c140f17e-45ef-4daf-8cd8-9292e756f50c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:56.026958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c140f17e-45ef-4daf-8cd8-9292e756f50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:56.083400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab4786-7133-4f73-9c69-3093adf7cb0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.084079 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab4786-7133-4f73-9c69-3093adf7cb0a service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.084364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab4786-7133-4f73-9c69-3093adf7cb0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.085393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab4786-7133-4f73-9c69-3093adf7cb0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.087666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-daab4786-7133-4f73-9c69-3093adf7cb0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:56.144273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1078/2132] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:56.145979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26cf73-3f58-457b-955a-76b275932a62 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.146270 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26cf73-3f58-457b-955a-76b275932a62 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.146544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26cf73-3f58-457b-955a-76b275932a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.182634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab4786-7133-4f73-9c69-3093adf7cb0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.184134 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab4786-7133-4f73-9c69-3093adf7cb0a service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:56.199719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4b26cf73-3f58-457b-955a-76b275932a62 None None] MFA Rules not processed for user `fa397bd1eff7495387deaf83428270b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:56.201393 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4b26cf73-3f58-457b-955a-76b275932a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:56.210314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1055/2133] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:56.211686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba007bb-dea4-4b07-b4b6-a3916c3f4ddf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.212043 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba007bb-dea4-4b07-b4b6-a3916c3f4ddf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.212332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba007bb-dea4-4b07-b4b6-a3916c3f4ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.237293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5ba007bb-dea4-4b07-b4b6-a3916c3f4ddf None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:56.239861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5ba007bb-dea4-4b07-b4b6-a3916c3f4ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:56.255789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1056/2134] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:56.259464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba007bb-dea4-4b07-b4b6-a3916c3f4ddf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:56.279881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba007bb-dea4-4b07-b4b6-a3916c3f4ddf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.285337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c7790-863f-4c31-8f08-066ccfacb708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.286286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c7790-863f-4c31-8f08-066ccfacb708 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.286669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c7790-863f-4c31-8f08-066ccfacb708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.287098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c7790-863f-4c31-8f08-066ccfacb708 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.287917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c7790-863f-4c31-8f08-066ccfacb708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:06:56.288813 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c7790-863f-4c31-8f08-066ccfacb708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.291238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c7790-863f-4c31-8f08-066ccfacb708 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:56.321671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1057/2135] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:06:56.326352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c7790-863f-4c31-8f08-066ccfacb708 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:56.344496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c7790-863f-4c31-8f08-066ccfacb708 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.350379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccdc99-525f-4ab9-b063-3c07933e5dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.351775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccdc99-525f-4ab9-b063-3c07933e5dce admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.352071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccdc99-525f-4ab9-b063-3c07933e5dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.352349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccdc99-525f-4ab9-b063-3c07933e5dce admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.353123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccdc99-525f-4ab9-b063-3c07933e5dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.355710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccdc99-525f-4ab9-b063-3c07933e5dce admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:56.362794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1079/2136] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:56.366080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26cf73-3f58-457b-955a-76b275932a62 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:56.368238 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4b26cf73-3f58-457b-955a-76b275932a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:56.389987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26cf73-3f58-457b-955a-76b275932a62 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.397200 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe2107-73e1-450f-a8a4-bf62c1fda3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.397851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe2107-73e1-450f-a8a4-bf62c1fda3bc admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.398136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe2107-73e1-450f-a8a4-bf62c1fda3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.398413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe2107-73e1-450f-a8a4-bf62c1fda3bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.399116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe2107-73e1-450f-a8a4-bf62c1fda3bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.401070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe2107-73e1-450f-a8a4-bf62c1fda3bc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:56.464382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1058/2137] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:56 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:56.468458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccdc99-525f-4ab9-b063-3c07933e5dce admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:56.534814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1080/2138] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:56 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:56.536434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe2107-73e1-450f-a8a4-bf62c1fda3bc admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:56.542799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccdc99-525f-4ab9-b063-3c07933e5dce admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.632375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe2107-73e1-450f-a8a4-bf62c1fda3bc admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.635318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5302e-6e49-462f-8b2b-3ce294997458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.635938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5302e-6e49-462f-8b2b-3ce294997458 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.636221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5302e-6e49-462f-8b2b-3ce294997458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.636495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5302e-6e49-462f-8b2b-3ce294997458 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.637359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5302e-6e49-462f-8b2b-3ce294997458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.639425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5302e-6e49-462f-8b2b-3ce294997458 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:56.645851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a69b3-1da2-4dd7-aa1f-ebf8baff2750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.646475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a69b3-1da2-4dd7-aa1f-ebf8baff2750 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.646775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a69b3-1da2-4dd7-aa1f-ebf8baff2750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.647420 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a69b3-1da2-4dd7-aa1f-ebf8baff2750 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.647997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a69b3-1da2-4dd7-aa1f-ebf8baff2750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.653178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a69b3-1da2-4dd7-aa1f-ebf8baff2750 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:56.780027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1059/2139] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:56 2022] POST /identity/v3/users => generated 284 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:56.780195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1081/2140] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:56 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:56.783098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5302e-6e49-462f-8b2b-3ce294997458 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:56.786540 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a69b3-1da2-4dd7-aa1f-ebf8baff2750 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:56.814498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a69b3-1da2-4dd7-aa1f-ebf8baff2750 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.815299 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5302e-6e49-462f-8b2b-3ce294997458 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.820861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473672e-0cf2-45f3-90af-73f7059cfc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.821547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473672e-0cf2-45f3-90af-73f7059cfc54 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.821853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473672e-0cf2-45f3-90af-73f7059cfc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.822381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473672e-0cf2-45f3-90af-73f7059cfc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.823148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473672e-0cf2-45f3-90af-73f7059cfc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.824799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473672e-0cf2-45f3-90af-73f7059cfc54 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:56.825492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d28502-e1bc-4c26-957f-dbc3adcf5c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.833468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d28502-e1bc-4c26-957f-dbc3adcf5c70 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.834069 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d28502-e1bc-4c26-957f-dbc3adcf5c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.837315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d28502-e1bc-4c26-957f-dbc3adcf5c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.837315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d28502-e1bc-4c26-957f-dbc3adcf5c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.837315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d28502-e1bc-4c26-957f-dbc3adcf5c70 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:56.838835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1082/2141] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:56.842940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473672e-0cf2-45f3-90af-73f7059cfc54 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:56.860184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1060/2142] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:06:56.862057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d28502-e1bc-4c26-957f-dbc3adcf5c70 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:56.870366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473672e-0cf2-45f3-90af-73f7059cfc54 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.881026 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d28502-e1bc-4c26-957f-dbc3adcf5c70 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.887462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d47c3-382f-4d3b-9e40-180a14535735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.889436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d47c3-382f-4d3b-9e40-180a14535735 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.889436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d47c3-382f-4d3b-9e40-180a14535735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.889436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d47c3-382f-4d3b-9e40-180a14535735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.890244 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d47c3-382f-4d3b-9e40-180a14535735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.894124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d47c3-382f-4d3b-9e40-180a14535735 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:56.897335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e60343-9d88-46e5-ad17-ab52ea32c348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.898034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60343-9d88-46e5-ad17-ab52ea32c348 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.898405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60343-9d88-46e5-ad17-ab52ea32c348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.898772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60343-9d88-46e5-ad17-ab52ea32c348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.899492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e60343-9d88-46e5-ad17-ab52ea32c348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.901834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e60343-9d88-46e5-ad17-ab52ea32c348 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:56.908898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1083/2143] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:56.913910 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d47c3-382f-4d3b-9e40-180a14535735 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:56.919374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1061/2144] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:56.923590 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e60343-9d88-46e5-ad17-ab52ea32c348 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:56.926365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d47c3-382f-4d3b-9e40-180a14535735 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.935625 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e60343-9d88-46e5-ad17-ab52ea32c348 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:56.936794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea5c87-3029-4c6f-8011-c92f7c5aa1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.938428 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea5c87-3029-4c6f-8011-c92f7c5aa1b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.938977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea5c87-3029-4c6f-8011-c92f7c5aa1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.939733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea5c87-3029-4c6f-8011-c92f7c5aa1b7 admin admin] PATH_INFO: `/v3/projects/11a05399e9954b9dbfc98436b0447a5f/users/f616eb935c424368900feea0a0eabdb9/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.941511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c430f-9987-4432-92d5-da49927f1c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:56.942244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c430f-9987-4432-92d5-da49927f1c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:56.942515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c430f-9987-4432-92d5-da49927f1c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:56.949207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c430f-9987-4432-92d5-da49927f1c9f admin admin] PATH_INFO: `/v3/projects/1a12314f7d5f419bb45e3564656553e3/users/edd09843082748a79d26452687f7a989/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:56.963393 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea5c87-3029-4c6f-8011-c92f7c5aa1b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11a05399e9954b9dbfc98436b0447a5f, user_id=f616eb935c424368900feea0a0eabdb9, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.964462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c430f-9987-4432-92d5-da49927f1c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a12314f7d5f419bb45e3564656553e3, user_id=edd09843082748a79d26452687f7a989, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:56.966401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea5c87-3029-4c6f-8011-c92f7c5aa1b7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:56.967352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c430f-9987-4432-92d5-da49927f1c9f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:57.078905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1084/2145] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:56 2022] PUT /identity/v3/projects/11a05399e9954b9dbfc98436b0447a5f/users/f616eb935c424368900feea0a0eabdb9/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:57.090779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c3541-d2da-4c7b-96aa-68bb8eb10e92 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:57.091957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c3541-d2da-4c7b-96aa-68bb8eb10e92 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:57.092739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c3541-d2da-4c7b-96aa-68bb8eb10e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:57.094772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:57.102187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1062/2146] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:56 2022] PUT /identity/v3/projects/1a12314f7d5f419bb45e3564656553e3/users/edd09843082748a79d26452687f7a989/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:57.113782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda33e-915b-4254-bc0b-c591b882e57d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:57.114076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda33e-915b-4254-bc0b-c591b882e57d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:57.114354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda33e-915b-4254-bc0b-c591b882e57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:57.157891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e2cda33e-915b-4254-bc0b-c591b882e57d None None] MFA Rules not processed for user `edd09843082748a79d26452687f7a989`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:57.159480 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e2cda33e-915b-4254-bc0b-c591b882e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:57.177901 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-117c3541-d2da-4c7b-96aa-68bb8eb10e92 None None] MFA Rules not processed for user `f616eb935c424368900feea0a0eabdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:57.184796 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-117c3541-d2da-4c7b-96aa-68bb8eb10e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:57.251112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1063/2147] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:06:57.265415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cda33e-915b-4254-bc0b-c591b882e57d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:57.267614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e2cda33e-915b-4254-bc0b-c591b882e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:57.288970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1085/2148] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:57.294095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c3541-d2da-4c7b-96aa-68bb8eb10e92 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:57.374238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cda33e-915b-4254-bc0b-c591b882e57d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:57.380867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c5849-16d3-4f87-a933-7cde292caffe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c214b2c97a7b41698b433eac8379442a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:57.381735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c5849-16d3-4f87-a933-7cde292caffe service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:57.382317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c5849-16d3-4f87-a933-7cde292caffe service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:57.382824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c5849-16d3-4f87-a933-7cde292caffe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:57.401268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c5849-16d3-4f87-a933-7cde292caffe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:57.403187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c5849-16d3-4f87-a933-7cde292caffe service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:57.410083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c3541-d2da-4c7b-96aa-68bb8eb10e92 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:57.415559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149471c7-17ae-48d3-8567-3996a7727b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:57.416262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149471c7-17ae-48d3-8567-3996a7727b97 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:57.416655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149471c7-17ae-48d3-8567-3996a7727b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:57.416935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149471c7-17ae-48d3-8567-3996a7727b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:57.420487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149471c7-17ae-48d3-8567-3996a7727b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:57.420487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149471c7-17ae-48d3-8567-3996a7727b97 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:57.423859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1064/2149] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:06:57.477641 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:57.533976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1086/2150] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:06:57 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:06:57.541386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c5849-16d3-4f87-a933-7cde292caffe service glance] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:57.564567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c5849-16d3-4f87-a933-7cde292caffe service glance] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:57.659085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5460e-f061-49da-9c84-32076ef12632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:57.659715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5460e-f061-49da-9c84-32076ef12632 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:57.659999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5460e-f061-49da-9c84-32076ef12632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:57.660273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5460e-f061-49da-9c84-32076ef12632 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:57.660945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5460e-f061-49da-9c84-32076ef12632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:57.662971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5460e-f061-49da-9c84-32076ef12632 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:57.755375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1065/2151] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:06:57 2022] POST /identity/v3/users => generated 299 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:06:57.765029 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149471c7-17ae-48d3-8567-3996a7727b97 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:57.786401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149471c7-17ae-48d3-8567-3996a7727b97 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:57.798990 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c6e88-8a61-4117-bd4f-62daa1a8971d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:57.803389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c6e88-8a61-4117-bd4f-62daa1a8971d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:57.803389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c6e88-8a61-4117-bd4f-62daa1a8971d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:57.803389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c6e88-8a61-4117-bd4f-62daa1a8971d admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:57.803389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c6e88-8a61-4117-bd4f-62daa1a8971d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:57.803389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c6e88-8a61-4117-bd4f-62daa1a8971d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:57.820571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1087/2152] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:57.826376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5460e-f061-49da-9c84-32076ef12632 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:57.854787 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5460e-f061-49da-9c84-32076ef12632 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:57.879204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ad885-e741-4cdc-a620-3b3260653d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:57.880509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ad885-e741-4cdc-a620-3b3260653d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:57.880792 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ad885-e741-4cdc-a620-3b3260653d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:57.881063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ad885-e741-4cdc-a620-3b3260653d91 admin admin] PATH_INFO: `/v3/projects/148d7ac088fb4e7ba4284981cf9cf326/users/50784564a8644275aad0cde2cc88610e/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:57.945692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ad885-e741-4cdc-a620-3b3260653d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=148d7ac088fb4e7ba4284981cf9cf326, user_id=50784564a8644275aad0cde2cc88610e, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:57.948467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ad885-e741-4cdc-a620-3b3260653d91 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:58.068493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1066/2153] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:57 2022] PUT /identity/v3/projects/148d7ac088fb4e7ba4284981cf9cf326/users/50784564a8644275aad0cde2cc88610e/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:58.072641 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c6e88-8a61-4117-bd4f-62daa1a8971d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:58.101236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c6e88-8a61-4117-bd4f-62daa1a8971d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:58.267906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe84f8-791d-4d8f-8d72-903efc6d8b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:58.268837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe84f8-791d-4d8f-8d72-903efc6d8b4b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:58.269121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe84f8-791d-4d8f-8d72-903efc6d8b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:58.269419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe84f8-791d-4d8f-8d72-903efc6d8b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:58.270751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe84f8-791d-4d8f-8d72-903efc6d8b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:58.272181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe84f8-791d-4d8f-8d72-903efc6d8b4b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:58.292789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1088/2154] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:06:58.294104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ad885-e741-4cdc-a620-3b3260653d91 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:58.316283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ad885-e741-4cdc-a620-3b3260653d91 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:58.337445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bd1c6-8f2a-4f37-aac3-e379bed06ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:58.338278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bd1c6-8f2a-4f37-aac3-e379bed06ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:58.338565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bd1c6-8f2a-4f37-aac3-e379bed06ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:58.339144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bd1c6-8f2a-4f37-aac3-e379bed06ae2 admin admin] PATH_INFO: `/v3/projects/148d7ac088fb4e7ba4284981cf9cf326/users/50784564a8644275aad0cde2cc88610e/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:58.342969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bd1c6-8f2a-4f37-aac3-e379bed06ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=148d7ac088fb4e7ba4284981cf9cf326, user_id=50784564a8644275aad0cde2cc88610e, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:58.352224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bd1c6-8f2a-4f37-aac3-e379bed06ae2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:58.370391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1067/2155] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:06:58 2022] PUT /identity/v3/projects/148d7ac088fb4e7ba4284981cf9cf326/users/50784564a8644275aad0cde2cc88610e/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:06:58.381679 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decbaa0-ec63-4b86-bd04-343fbdf8314c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:58.382405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decbaa0-ec63-4b86-bd04-343fbdf8314c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:58.382712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decbaa0-ec63-4b86-bd04-343fbdf8314c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:58.462388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6decbaa0-ec63-4b86-bd04-343fbdf8314c None None] MFA Rules not processed for user `50784564a8644275aad0cde2cc88610e`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:58.464434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6decbaa0-ec63-4b86-bd04-343fbdf8314c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:58.591912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1089/2156] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:58.709928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff2601-d44c-4af9-9517-b67a57fb0aa5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:58.710219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff2601-d44c-4af9-9517-b67a57fb0aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:58.710494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff2601-d44c-4af9-9517-b67a57fb0aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:58.801267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ffff2601-d44c-4af9-9517-b67a57fb0aa5 None None] MFA Rules not processed for user `81e89f8fb70d49d18cd0c81c98dd4add`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:06:58.801267 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ffff2601-d44c-4af9-9517-b67a57fb0aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:58.866274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decbaa0-ec63-4b86-bd04-343fbdf8314c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:58.868493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6decbaa0-ec63-4b86-bd04-343fbdf8314c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:58.934829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1068/2157] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:06:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:06:58.949484 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff2601-d44c-4af9-9517-b67a57fb0aa5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:06:58.951383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ffff2601-d44c-4af9-9517-b67a57fb0aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:06:58.957398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decbaa0-ec63-4b86-bd04-343fbdf8314c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:58.962090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2422e-5451-4065-9bc8-518822498c15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:58.962735 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2422e-5451-4065-9bc8-518822498c15 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:58.963328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2422e-5451-4065-9bc8-518822498c15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:58.963865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2422e-5451-4065-9bc8-518822498c15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:59.032058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2422e-5451-4065-9bc8-518822498c15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:59.034255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2422e-5451-4065-9bc8-518822498c15 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:59.045017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1090/2158] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:59.068303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff2601-d44c-4af9-9517-b67a57fb0aa5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:06:59.074455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc47cc-d444-45d7-9e5e-2bdad2e820fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:06:59.075632 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc47cc-d444-45d7-9e5e-2bdad2e820fa service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:06:59.075964 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc47cc-d444-45d7-9e5e-2bdad2e820fa service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:06:59.076238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc47cc-d444-45d7-9e5e-2bdad2e820fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:06:59.089746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc47cc-d444-45d7-9e5e-2bdad2e820fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:06:59.091580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc47cc-d444-45d7-9e5e-2bdad2e820fa service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:06:59.103685 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1069/2159] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:06:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:06:59.131466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:06:59.538938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:00.684851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:01.978804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:02.520329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f143d4-988c-4c51-bc8d-98a4e402bdc1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:02.526282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f143d4-988c-4c51-bc8d-98a4e402bdc1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:02.526282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f143d4-988c-4c51-bc8d-98a4e402bdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:02.538116 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8ec1c-aba6-4bf0-aea5-f98308a5bf13 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:02.538809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8ec1c-aba6-4bf0-aea5-f98308a5bf13 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:02.538809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8ec1c-aba6-4bf0-aea5-f98308a5bf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:02.563602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-99f8ec1c-aba6-4bf0-aea5-f98308a5bf13 None None] MFA Rules not processed for user `81e89f8fb70d49d18cd0c81c98dd4add`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:02.565326 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-99f8ec1c-aba6-4bf0-aea5-f98308a5bf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:02.575940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1070/2160] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:07:02.588878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:02.597023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d3f143d4-988c-4c51-bc8d-98a4e402bdc1 None None] MFA Rules not processed for user `e4107a0136834061828fb975136de88e`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:02.597023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d3f143d4-988c-4c51-bc8d-98a4e402bdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:02.693538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8ec1c-aba6-4bf0-aea5-f98308a5bf13 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:02.713119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8ec1c-aba6-4bf0-aea5-f98308a5bf13 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:02.720948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af0ff7-b4d1-4165-852e-f5c4a9affcec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:02.721842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0ff7-b4d1-4165-852e-f5c4a9affcec service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:02.722300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0ff7-b4d1-4165-852e-f5c4a9affcec service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:02.723104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0ff7-b4d1-4165-852e-f5c4a9affcec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:02.725842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-67af0ff7-b4d1-4165-852e-f5c4a9affcec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:02.761958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1091/2161] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:02.776011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f143d4-988c-4c51-bc8d-98a4e402bdc1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:02.866364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af0ff7-b4d1-4165-852e-f5c4a9affcec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:02.869470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af0ff7-b4d1-4165-852e-f5c4a9affcec service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:02.890588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1071/2162] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:02.982550 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f143d4-988c-4c51-bc8d-98a4e402bdc1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:02.996724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28f398-8625-432c-92b8-e32f51c1172c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c214b2c97a7b41698b433eac8379442a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:02.999506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28f398-8625-432c-92b8-e32f51c1172c service glance] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:03.000148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28f398-8625-432c-92b8-e32f51c1172c service glance] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:03.001579 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28f398-8625-432c-92b8-e32f51c1172c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:03.007068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1f28f398-8625-432c-92b8-e32f51c1172c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:03.186305 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28f398-8625-432c-92b8-e32f51c1172c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:03.189630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28f398-8625-432c-92b8-e32f51c1172c service glance] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:03.211407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1092/2163] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:03.717238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39f8f2-599a-466f-8507-4bb5110fb27e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:03.718214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39f8f2-599a-466f-8507-4bb5110fb27e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:03.718670 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39f8f2-599a-466f-8507-4bb5110fb27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:03.772564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28f398-8625-432c-92b8-e32f51c1172c service glance] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:03.788488 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1a39f8f2-599a-466f-8507-4bb5110fb27e None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:03.788638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28f398-8625-432c-92b8-e32f51c1172c service glance] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:03.788903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1a39f8f2-599a-466f-8507-4bb5110fb27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:03.818567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dabb8-2352-4318-a6dc-d829c881c35c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:03.819004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dabb8-2352-4318-a6dc-d829c881c35c service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:03.822446 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dabb8-2352-4318-a6dc-d829c881c35c service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:03.822952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dabb8-2352-4318-a6dc-d829c881c35c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:03.922115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1072/2164] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:07:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:04.009566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0dabb8-2352-4318-a6dc-d829c881c35c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:04.011337 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0dabb8-2352-4318-a6dc-d829c881c35c service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:04.019811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39f8f2-599a-466f-8507-4bb5110fb27e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:04.022095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1a39f8f2-599a-466f-8507-4bb5110fb27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:04.022251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1093/2165] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:04.188771 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39f8f2-599a-466f-8507-4bb5110fb27e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:04.197667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470171f5-eeca-496b-b33e-d010791f3b13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:04.198855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470171f5-eeca-496b-b33e-d010791f3b13 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:04.199768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470171f5-eeca-496b-b33e-d010791f3b13 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:04.200153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470171f5-eeca-496b-b33e-d010791f3b13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:04.223813 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470171f5-eeca-496b-b33e-d010791f3b13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:04.226125 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470171f5-eeca-496b-b33e-d010791f3b13 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:04.238216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1073/2166] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:07:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 16 08:07:04.408051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:04.597121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:06.610233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:07.369949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dabb8-2352-4318-a6dc-d829c881c35c service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:07.383847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dabb8-2352-4318-a6dc-d829c881c35c service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:07.389872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811b7e3-8fd8-401e-a267-0f645cf3f7fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:07.389872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811b7e3-8fd8-401e-a267-0f645cf3f7fa service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:07.389872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811b7e3-8fd8-401e-a267-0f645cf3f7fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:07.389872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811b7e3-8fd8-401e-a267-0f645cf3f7fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:07.406588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811b7e3-8fd8-401e-a267-0f645cf3f7fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:07.408941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811b7e3-8fd8-401e-a267-0f645cf3f7fa service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:07.422641 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1094/2167] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:07.444536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:08.040163 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:10.607109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:11.846505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:13.449781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:14.086510 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:16.411192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470171f5-eeca-496b-b33e-d010791f3b13 service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:16.484241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470171f5-eeca-496b-b33e-d010791f3b13 service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:16.490637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:16.491510 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:16.491788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:16.492101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] PATH_INFO: `/v3/users/fa397bd1eff7495387deaf83428270b8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:16.493680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa397bd1eff7495387deaf83428270b8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:16.495889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:16.780106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa397bd1eff7495387deaf83428270b8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:16.794268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa397bd1eff7495387deaf83428270b8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:16.802523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa397bd1eff7495387deaf83428270b8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:16.928330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa397bd1eff7495387deaf83428270b8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:16.929259 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa397bd1eff7495387deaf83428270b8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:16.938321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1074/2168] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:07:16 2022] DELETE /identity/v3/users/fa397bd1eff7495387deaf83428270b8 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:07:16.950298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292592-7559-449a-a646-f405b17b8d94 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:16.951354 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292592-7559-449a-a646-f405b17b8d94 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:16.951890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292592-7559-449a-a646-f405b17b8d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:16.973498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e9292592-7559-449a-a646-f405b17b8d94 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:16.974906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e9292592-7559-449a-a646-f405b17b8d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:17.039482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1095/2169] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:17.048594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:17.052387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:17.074013 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:17.110654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d0cb2-7070-403c-ac24-98cf0bd43507 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:17.115444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece2b59-0cab-43b7-ad52-654630a81a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:17.116045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece2b59-0cab-43b7-ad52-654630a81a3e service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:17.116533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece2b59-0cab-43b7-ad52-654630a81a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:17.116806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece2b59-0cab-43b7-ad52-654630a81a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:17.127311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece2b59-0cab-43b7-ad52-654630a81a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:17.128752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece2b59-0cab-43b7-ad52-654630a81a3e service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:17.138482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1075/2170] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:17.646037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deafef-6df9-44b6-8e75-a47cf2732246 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:17.650350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deafef-6df9-44b6-8e75-a47cf2732246 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:17.650350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deafef-6df9-44b6-8e75-a47cf2732246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:17.669427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a4deafef-6df9-44b6-8e75-a47cf2732246 None None] MFA Rules not processed for user `66702831c1ca4158976920f6be2cc80c`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:17.670698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a4deafef-6df9-44b6-8e75-a47cf2732246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:17.676681 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece2b59-0cab-43b7-ad52-654630a81a3e service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:17.694797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bece2b59-0cab-43b7-ad52-654630a81a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:17.697027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:17.733342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece2b59-0cab-43b7-ad52-654630a81a3e service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:17.745386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:17.745991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1096/2171] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:17.747841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:17.748196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:17.748548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] PATH_INFO: `/v3/projects/9bf83d3dec5849619a9b687124cd11ac` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:17.749939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf83d3dec5849619a9b687124cd11ac)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:17.758196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:17.889920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf83d3dec5849619a9b687124cd11ac', 'request_id': 'req-619be34e-17b3-48f9-b213-9bb61d714af7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:17.897705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf83d3dec5849619a9b687124cd11ac', 'request_id': 'req-619be34e-17b3-48f9-b213-9bb61d714af7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:17.904938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf83d3dec5849619a9b687124cd11ac', 'request_id': 'req-619be34e-17b3-48f9-b213-9bb61d714af7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:18.052916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] The token cache is being invalidate because project 9bf83d3dec5849619a9b687124cd11ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:07:18.056709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:18.056709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:18.056709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1076/2172] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:07:17 2022] DELETE /identity/v3/projects/9bf83d3dec5849619a9b687124cd11ac => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:18.059950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deafef-6df9-44b6-8e75-a47cf2732246 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:18.072264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deafef-6df9-44b6-8e75-a47cf2732246 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:18.149362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:18.150383 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:18.151105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:18.151534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] PATH_INFO: `/v3/users/f616eb935c424368900feea0a0eabdb9` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:18.152881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f616eb935c424368900feea0a0eabdb9)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:18.155572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:18.429179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f616eb935c424368900feea0a0eabdb9'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:18.457765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f616eb935c424368900feea0a0eabdb9'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:18.466997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f616eb935c424368900feea0a0eabdb9'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:18.586947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f616eb935c424368900feea0a0eabdb9'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:18.588151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f616eb935c424368900feea0a0eabdb9'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:18.610087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1097/2173] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:07:18 2022] DELETE /identity/v3/users/f616eb935c424368900feea0a0eabdb9 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:19.261266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:19.261266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:19.285224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619be34e-17b3-48f9-b213-9bb61d714af7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:19.350547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:19.351405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:19.351705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:19.351984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] PATH_INFO: `/v3/projects/11a05399e9954b9dbfc98436b0447a5f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:19.353141 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11a05399e9954b9dbfc98436b0447a5f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:19.355175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:19.486527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:19.488738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11a05399e9954b9dbfc98436b0447a5f', 'request_id': 'req-8924abf0-d652-43a8-ada3-3545b141aa55'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:19.498103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11a05399e9954b9dbfc98436b0447a5f', 'request_id': 'req-8924abf0-d652-43a8-ada3-3545b141aa55'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:19.506615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11a05399e9954b9dbfc98436b0447a5f', 'request_id': 'req-8924abf0-d652-43a8-ada3-3545b141aa55'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:19.616023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] The token cache is being invalidate because project 11a05399e9954b9dbfc98436b0447a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:07:19.625705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:19.625705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:19.625705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1077/2174] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:07:19 2022] DELETE /identity/v3/projects/11a05399e9954b9dbfc98436b0447a5f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:19.626714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:19.626714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:19.641555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6799c49-a1ee-46a4-8174-9051a2b6c29a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:19.690212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:19.700621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495ff11-5ebd-43ee-9d02-b301f0e25f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:19.701261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495ff11-5ebd-43ee-9d02-b301f0e25f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:19.701547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495ff11-5ebd-43ee-9d02-b301f0e25f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:19.701823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495ff11-5ebd-43ee-9d02-b301f0e25f7e admin admin] PATH_INFO: `/v3/users/50784564a8644275aad0cde2cc88610e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:19.702985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495ff11-5ebd-43ee-9d02-b301f0e25f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=50784564a8644275aad0cde2cc88610e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:19.704940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495ff11-5ebd-43ee-9d02-b301f0e25f7e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:19.940991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0495ff11-5ebd-43ee-9d02-b301f0e25f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50784564a8644275aad0cde2cc88610e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:19.952993 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0495ff11-5ebd-43ee-9d02-b301f0e25f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50784564a8644275aad0cde2cc88610e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:19.961081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0495ff11-5ebd-43ee-9d02-b301f0e25f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50784564a8644275aad0cde2cc88610e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:20.071909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0495ff11-5ebd-43ee-9d02-b301f0e25f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50784564a8644275aad0cde2cc88610e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:20.072556 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0495ff11-5ebd-43ee-9d02-b301f0e25f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50784564a8644275aad0cde2cc88610e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:20.079624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1098/2175] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:07:19 2022] DELETE /identity/v3/users/50784564a8644275aad0cde2cc88610e => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:20.531782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:20.535461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:20.554225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924abf0-d652-43a8-ada3-3545b141aa55 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:20.646481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:20.647151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:20.647458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:20.647728 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] PATH_INFO: `/v3/projects/148d7ac088fb4e7ba4284981cf9cf326` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:20.648839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] RBAC: Authorizing `identity:delete_project(project_id=148d7ac088fb4e7ba4284981cf9cf326)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:20.650829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:20.762233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '148d7ac088fb4e7ba4284981cf9cf326', 'request_id': 'req-98501a46-dca9-425a-9bbc-97977439560d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:20.769095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '148d7ac088fb4e7ba4284981cf9cf326', 'request_id': 'req-98501a46-dca9-425a-9bbc-97977439560d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:20.775688 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '148d7ac088fb4e7ba4284981cf9cf326', 'request_id': 'req-98501a46-dca9-425a-9bbc-97977439560d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:20.838338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] The token cache is being invalidate because project 148d7ac088fb4e7ba4284981cf9cf326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:07:20.838546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:20.839608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-98501a46-dca9-425a-9bbc-97977439560d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:20.840747 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1078/2176] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:07:20 2022] DELETE /identity/v3/projects/148d7ac088fb4e7ba4284981cf9cf326 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:20.845702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0891a-54a4-4b50-9012-188cba52619e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:20.846130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0891a-54a4-4b50-9012-188cba52619e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:20.846231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0891a-54a4-4b50-9012-188cba52619e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:20.865495 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a0f0891a-54a4-4b50-9012-188cba52619e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:20.871459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a0f0891a-54a4-4b50-9012-188cba52619e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:20.931713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1099/2177] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:07:20.934847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a44b5d-adad-4e0b-a65d-8e96003436dc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:20.935149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a44b5d-adad-4e0b-a65d-8e96003436dc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:20.935426 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a44b5d-adad-4e0b-a65d-8e96003436dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:20.957258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-80a44b5d-adad-4e0b-a65d-8e96003436dc None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:20.958479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-80a44b5d-adad-4e0b-a65d-8e96003436dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:20.966879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1079/2178] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:20.970902 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0891a-54a4-4b50-9012-188cba52619e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:20.981386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0891a-54a4-4b50-9012-188cba52619e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:20.994525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de31be5-6096-4dcf-a11a-4ed687fa92a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:20.995479 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de31be5-6096-4dcf-a11a-4ed687fa92a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:20.995599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de31be5-6096-4dcf-a11a-4ed687fa92a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:20.995901 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de31be5-6096-4dcf-a11a-4ed687fa92a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:20.996990 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de31be5-6096-4dcf-a11a-4ed687fa92a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:07:20.997162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de31be5-6096-4dcf-a11a-4ed687fa92a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:20.998878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de31be5-6096-4dcf-a11a-4ed687fa92a6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:21.011980 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1100/2179] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:07:21.016307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a44b5d-adad-4e0b-a65d-8e96003436dc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:21.025989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a44b5d-adad-4e0b-a65d-8e96003436dc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:21.031625 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85e6d2-1e81-45fc-92f5-c5633fff05a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:21.036780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85e6d2-1e81-45fc-92f5-c5633fff05a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:21.036780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85e6d2-1e81-45fc-92f5-c5633fff05a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:21.036780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85e6d2-1e81-45fc-92f5-c5633fff05a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:21.036780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85e6d2-1e81-45fc-92f5-c5633fff05a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:21.036780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85e6d2-1e81-45fc-92f5-c5633fff05a2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:21.133698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1080/2180] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:07:21 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:07:21.136166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de31be5-6096-4dcf-a11a-4ed687fa92a6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:21.140014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeba64-3628-4628-b5d4-7268752b2110 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:21.140014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeba64-3628-4628-b5d4-7268752b2110 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:21.140014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeba64-3628-4628-b5d4-7268752b2110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:21.146614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de31be5-6096-4dcf-a11a-4ed687fa92a6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:21.158363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ebdeba64-3628-4628-b5d4-7268752b2110 None None] MFA Rules not processed for user `ea2f6b804e164dbcb4f67926f23fdd59`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:21.159614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ebdeba64-3628-4628-b5d4-7268752b2110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:21.195576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:21.202973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:21.209021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fff287-1567-4505-8ae8-19241dc911a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:21.209709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fff287-1567-4505-8ae8-19241dc911a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:21.210086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fff287-1567-4505-8ae8-19241dc911a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:21.210426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fff287-1567-4505-8ae8-19241dc911a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:21.211146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fff287-1567-4505-8ae8-19241dc911a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:21.213109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fff287-1567-4505-8ae8-19241dc911a0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:21.217859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1081/2181] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:21.338577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1101/2182] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:07:21 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:07:21.339860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeba64-3628-4628-b5d4-7268752b2110 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:21.362633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeba64-3628-4628-b5d4-7268752b2110 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:21.372236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaee66c-d393-4252-8fd4-b8524bf87585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:21.372236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaee66c-d393-4252-8fd4-b8524bf87585 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:21.372236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaee66c-d393-4252-8fd4-b8524bf87585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:21.372236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaee66c-d393-4252-8fd4-b8524bf87585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:21.372680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaee66c-d393-4252-8fd4-b8524bf87585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:21.372680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaee66c-d393-4252-8fd4-b8524bf87585 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:21.382291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1082/2183] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:07:21.386425 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fff287-1567-4505-8ae8-19241dc911a0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:21.398133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fff287-1567-4505-8ae8-19241dc911a0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:21.403628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070be4c2-69a2-43d4-8e73-228b50ff8eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:21.405250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070be4c2-69a2-43d4-8e73-228b50ff8eab admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:21.405531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070be4c2-69a2-43d4-8e73-228b50ff8eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:21.406311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070be4c2-69a2-43d4-8e73-228b50ff8eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:21.406492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070be4c2-69a2-43d4-8e73-228b50ff8eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:21.408692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070be4c2-69a2-43d4-8e73-228b50ff8eab admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:21.423087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1102/2184] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:07:21.425087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaee66c-d393-4252-8fd4-b8524bf87585 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:21.436409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaee66c-d393-4252-8fd4-b8524bf87585 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:21.441591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b32c47-8c9c-482c-a927-990813e3e93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:21.442248 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b32c47-8c9c-482c-a927-990813e3e93c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:21.442525 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b32c47-8c9c-482c-a927-990813e3e93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:21.442848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b32c47-8c9c-482c-a927-990813e3e93c admin admin] PATH_INFO: `/v3/projects/14c55f15cadb4aaf80edb560ba3ea821/users/f6ed499e1f0049cc8c0b21bdb6c6943e/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:21.459968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b32c47-8c9c-482c-a927-990813e3e93c admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c55f15cadb4aaf80edb560ba3ea821, user_id=f6ed499e1f0049cc8c0b21bdb6c6943e, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:21.462899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b32c47-8c9c-482c-a927-990813e3e93c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:21.569607 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1083/2185] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:07:21 2022] PUT /identity/v3/projects/14c55f15cadb4aaf80edb560ba3ea821/users/f6ed499e1f0049cc8c0b21bdb6c6943e/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:21.580624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613920e-1856-45e9-bcd1-9305c4e5825c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:21.580978 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613920e-1856-45e9-bcd1-9305c4e5825c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:21.581323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613920e-1856-45e9-bcd1-9305c4e5825c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:21.610253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5613920e-1856-45e9-bcd1-9305c4e5825c None None] MFA Rules not processed for user `f6ed499e1f0049cc8c0b21bdb6c6943e`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:21.611711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5613920e-1856-45e9-bcd1-9305c4e5825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:21.696313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1103/2186] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:21.711774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b32c47-8c9c-482c-a927-990813e3e93c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:21.714390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-41b32c47-8c9c-482c-a927-990813e3e93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:21.827852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b32c47-8c9c-482c-a927-990813e3e93c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:21.838039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b3b38-deae-4dcb-9a69-0d4406b8bda4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:21.838039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b3b38-deae-4dcb-9a69-0d4406b8bda4 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:21.838039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b3b38-deae-4dcb-9a69-0d4406b8bda4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:21.838039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b3b38-deae-4dcb-9a69-0d4406b8bda4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:21.846949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b3b38-deae-4dcb-9a69-0d4406b8bda4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:21.849620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b3b38-deae-4dcb-9a69-0d4406b8bda4 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:21.870026 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1084/2187] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:24.994305 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09085330-291a-4ff1-b060-2e9009658033 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:24.994805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09085330-291a-4ff1-b060-2e9009658033 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:24.995015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09085330-291a-4ff1-b060-2e9009658033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:25.028175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-09085330-291a-4ff1-b060-2e9009658033 None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:25.029426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-09085330-291a-4ff1-b060-2e9009658033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:25.106621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1104/2188] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:07:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:25.122277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b3b38-deae-4dcb-9a69-0d4406b8bda4 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:25.123936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-253b3b38-deae-4dcb-9a69-0d4406b8bda4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:25.204259 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b3b38-deae-4dcb-9a69-0d4406b8bda4 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:25.209403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1572b-b9a3-4796-825f-fc9f68f63c47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:25.210012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1572b-b9a3-4796-825f-fc9f68f63c47 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:25.210293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1572b-b9a3-4796-825f-fc9f68f63c47 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:25.210571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1572b-b9a3-4796-825f-fc9f68f63c47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:25.218918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1572b-b9a3-4796-825f-fc9f68f63c47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:25.220648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1572b-b9a3-4796-825f-fc9f68f63c47 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:25.229106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1085/2189] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:07:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:07:25.351673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09085330-291a-4ff1-b060-2e9009658033 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:25.416090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09085330-291a-4ff1-b060-2e9009658033 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:25.420949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:25.427343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:25.427676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:25.430195 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] PATH_INFO: `/v3/users/f6ed499e1f0049cc8c0b21bdb6c6943e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:25.434306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6ed499e1f0049cc8c0b21bdb6c6943e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:25.436863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:25.462829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:25.670024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6ed499e1f0049cc8c0b21bdb6c6943e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:25.682855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6ed499e1f0049cc8c0b21bdb6c6943e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:25.689882 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6ed499e1f0049cc8c0b21bdb6c6943e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:25.757175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6ed499e1f0049cc8c0b21bdb6c6943e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:25.757814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6ed499e1f0049cc8c0b21bdb6c6943e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:25.764373 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1105/2190] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:07:25 2022] DELETE /identity/v3/users/f6ed499e1f0049cc8c0b21bdb6c6943e => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:07:25.769276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8593be-5f15-4799-8327-fa3c0d202525 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:25.769568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8593be-5f15-4799-8327-fa3c0d202525 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:25.769868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8593be-5f15-4799-8327-fa3c0d202525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:25.789856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2f8593be-5f15-4799-8327-fa3c0d202525 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:25.791124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2f8593be-5f15-4799-8327-fa3c0d202525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:25.825650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:25.855019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1086/2191] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:25.864681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:25.866714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:25.923992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b203d-6cbf-4306-b764-8268aa5cc61a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:25.927605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea288e0-d23a-406a-a78b-c3af5e884cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:25.928209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea288e0-d23a-406a-a78b-c3af5e884cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:25.928492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea288e0-d23a-406a-a78b-c3af5e884cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:25.928767 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea288e0-d23a-406a-a78b-c3af5e884cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:25.937868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea288e0-d23a-406a-a78b-c3af5e884cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:25.939325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea288e0-d23a-406a-a78b-c3af5e884cb4 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:25.949138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1106/2192] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:07:26.510470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8593be-5f15-4799-8327-fa3c0d202525 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:26.512122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2f8593be-5f15-4799-8327-fa3c0d202525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:26.523852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8593be-5f15-4799-8327-fa3c0d202525 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:26.528741 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:26.529368 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:26.529649 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:26.529929 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] PATH_INFO: `/v3/projects/14c55f15cadb4aaf80edb560ba3ea821` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:26.531691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14c55f15cadb4aaf80edb560ba3ea821)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:26.533905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:26.666478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14c55f15cadb4aaf80edb560ba3ea821', 'request_id': 'req-6018fc38-724f-4aea-987d-bf2c324c7d19'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:26.676162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14c55f15cadb4aaf80edb560ba3ea821', 'request_id': 'req-6018fc38-724f-4aea-987d-bf2c324c7d19'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:26.682910 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14c55f15cadb4aaf80edb560ba3ea821', 'request_id': 'req-6018fc38-724f-4aea-987d-bf2c324c7d19'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:26.759866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] The token cache is being invalidate because project 14c55f15cadb4aaf80edb560ba3ea821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:07:26.760164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:26.760842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6018fc38-724f-4aea-987d-bf2c324c7d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:26.762863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1087/2193] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:07:26 2022] DELETE /identity/v3/projects/14c55f15cadb4aaf80edb560ba3ea821 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:26.767433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9065-902d-442f-93c4-9e9a6f42f3a4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:26.767947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9065-902d-442f-93c4-9e9a6f42f3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:26.768051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9065-902d-442f-93c4-9e9a6f42f3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:26.789430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9e4a9065-902d-442f-93c4-9e9a6f42f3a4 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:26.790877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9e4a9065-902d-442f-93c4-9e9a6f42f3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:26.851803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1107/2194] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:26.859477 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d5555-1684-4e24-9e03-71628ddf8160 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:26.859760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d5555-1684-4e24-9e03-71628ddf8160 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:26.860033 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d5555-1684-4e24-9e03-71628ddf8160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:26.883384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f67d5555-1684-4e24-9e03-71628ddf8160 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:26.884597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f67d5555-1684-4e24-9e03-71628ddf8160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:26.898335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1088/2195] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:26.899767 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a9065-902d-442f-93c4-9e9a6f42f3a4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:26.911926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a9065-902d-442f-93c4-9e9a6f42f3a4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:26.916860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bdb7c-e725-4948-8d58-ee1cbab90404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:26.917471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bdb7c-e725-4948-8d58-ee1cbab90404 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:26.917752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bdb7c-e725-4948-8d58-ee1cbab90404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:26.918025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bdb7c-e725-4948-8d58-ee1cbab90404 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:26.918713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bdb7c-e725-4948-8d58-ee1cbab90404 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:07:26.919010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bdb7c-e725-4948-8d58-ee1cbab90404 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:26.920626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bdb7c-e725-4948-8d58-ee1cbab90404 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:26.933439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1108/2196] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:07:26.937699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d5555-1684-4e24-9e03-71628ddf8160 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:26.946962 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d5555-1684-4e24-9e03-71628ddf8160 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:26.951598 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f240cba-04ea-4837-b16d-5e6e02822f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:26.952191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f240cba-04ea-4837-b16d-5e6e02822f93 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:26.952467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f240cba-04ea-4837-b16d-5e6e02822f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:26.952735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f240cba-04ea-4837-b16d-5e6e02822f93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:26.953366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f240cba-04ea-4837-b16d-5e6e02822f93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:26.955302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f240cba-04ea-4837-b16d-5e6e02822f93 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:27.027425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1089/2197] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:07:26 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:07:27.031784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bdb7c-e725-4948-8d58-ee1cbab90404 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:27.042806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bdb7c-e725-4948-8d58-ee1cbab90404 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:27.059719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:27.099715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae8acf-0b23-43df-a410-c3e262094232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:27.100351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae8acf-0b23-43df-a410-c3e262094232 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:27.100630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae8acf-0b23-43df-a410-c3e262094232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:27.100903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae8acf-0b23-43df-a410-c3e262094232 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:27.101539 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae8acf-0b23-43df-a410-c3e262094232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:27.103596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae8acf-0b23-43df-a410-c3e262094232 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:27.191556 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1109/2198] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:07:27 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:07:27.195509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f240cba-04ea-4837-b16d-5e6e02822f93 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:27.205962 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f240cba-04ea-4837-b16d-5e6e02822f93 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:27.211150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54c1c0-264e-4b0f-b9b5-df46829cc53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:27.211746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54c1c0-264e-4b0f-b9b5-df46829cc53e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:27.212027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54c1c0-264e-4b0f-b9b5-df46829cc53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:27.212300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54c1c0-264e-4b0f-b9b5-df46829cc53e admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:27.212942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54c1c0-264e-4b0f-b9b5-df46829cc53e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:27.214529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54c1c0-264e-4b0f-b9b5-df46829cc53e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:27.225312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1090/2199] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:07:27.229082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae8acf-0b23-43df-a410-c3e262094232 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:27.238963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae8acf-0b23-43df-a410-c3e262094232 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:27.243737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e58bc-fcd3-4eb1-bc8d-38f7fc12753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:27.244334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e58bc-fcd3-4eb1-bc8d-38f7fc12753d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:27.244618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e58bc-fcd3-4eb1-bc8d-38f7fc12753d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:27.244898 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e58bc-fcd3-4eb1-bc8d-38f7fc12753d admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:27.245566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e58bc-fcd3-4eb1-bc8d-38f7fc12753d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:27.247198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e58bc-fcd3-4eb1-bc8d-38f7fc12753d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:27.257808 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1110/2200] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:07:27.261643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54c1c0-264e-4b0f-b9b5-df46829cc53e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:27.271809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54c1c0-264e-4b0f-b9b5-df46829cc53e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:27.276367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3303b5-f495-4708-9e7c-11a8ffec6c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:27.277003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3303b5-f495-4708-9e7c-11a8ffec6c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:27.277284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3303b5-f495-4708-9e7c-11a8ffec6c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:27.277560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3303b5-f495-4708-9e7c-11a8ffec6c04 admin admin] PATH_INFO: `/v3/projects/0f74dfa55d37442996d298d4df1d7734/users/fa542d7bf6134efb99f25456d852f600/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:27.292516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3303b5-f495-4708-9e7c-11a8ffec6c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f74dfa55d37442996d298d4df1d7734, user_id=fa542d7bf6134efb99f25456d852f600, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:27.295191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3303b5-f495-4708-9e7c-11a8ffec6c04 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:27.382466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1091/2201] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:07:27 2022] PUT /identity/v3/projects/0f74dfa55d37442996d298d4df1d7734/users/fa542d7bf6134efb99f25456d852f600/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:27.392902 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9ec6c-72d2-43a0-b625-4b0e204543f5 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:27.393235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9ec6c-72d2-43a0-b625-4b0e204543f5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:27.393579 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9ec6c-72d2-43a0-b625-4b0e204543f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:27.422306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5ff9ec6c-72d2-43a0-b625-4b0e204543f5 None None] MFA Rules not processed for user `fa542d7bf6134efb99f25456d852f600`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:27.423573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5ff9ec6c-72d2-43a0-b625-4b0e204543f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:27.498341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1111/2202] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:27.511423 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3303b5-f495-4708-9e7c-11a8ffec6c04 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:27.513787 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7e3303b5-f495-4708-9e7c-11a8ffec6c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:27.576918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3303b5-f495-4708-9e7c-11a8ffec6c04 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:27.580617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41a18c-f807-45fb-8d66-6a99d6deb5e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:27.581804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41a18c-f807-45fb-8d66-6a99d6deb5e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:27.582087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41a18c-f807-45fb-8d66-6a99d6deb5e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:27.582362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41a18c-f807-45fb-8d66-6a99d6deb5e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:27.591098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41a18c-f807-45fb-8d66-6a99d6deb5e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:27.592537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41a18c-f807-45fb-8d66-6a99d6deb5e0 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:27.601603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1092/2203] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:37.495992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:37.598083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:40.325090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9ec6c-72d2-43a0-b625-4b0e204543f5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:40.326833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5ff9ec6c-72d2-43a0-b625-4b0e204543f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:40.407487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9ec6c-72d2-43a0-b625-4b0e204543f5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:40.412072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:40.412723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:40.413002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:40.413272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] PATH_INFO: `/v3/users/f62ac79d3db74b7fb0dbc901a95c9e73` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:40.414513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f62ac79d3db74b7fb0dbc901a95c9e73)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:40.416768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:40.631040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f62ac79d3db74b7fb0dbc901a95c9e73'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:40.649328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f62ac79d3db74b7fb0dbc901a95c9e73'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:40.656875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f62ac79d3db74b7fb0dbc901a95c9e73'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:40.714873 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f62ac79d3db74b7fb0dbc901a95c9e73'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:40.715514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f62ac79d3db74b7fb0dbc901a95c9e73'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:40.730298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1112/2204] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:07:40 2022] DELETE /identity/v3/users/f62ac79d3db74b7fb0dbc901a95c9e73 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:40.732558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21caae-063b-47cb-838f-e43c5a0ca419 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:40.733438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21caae-063b-47cb-838f-e43c5a0ca419 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:40.733926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21caae-063b-47cb-838f-e43c5a0ca419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:40.763858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6e21caae-063b-47cb-838f-e43c5a0ca419 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:40.765391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6e21caae-063b-47cb-838f-e43c5a0ca419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:40.841850 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1093/2205] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:40.852536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:40.854110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:40.931527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d0107-7a84-4fd7-a0c7-ed621a4f0959 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:40.935791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ced438-82e0-45d1-bb7b-5c2ba88d380e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:40.936405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ced438-82e0-45d1-bb7b-5c2ba88d380e service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:40.936841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ced438-82e0-45d1-bb7b-5c2ba88d380e service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:40.936959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ced438-82e0-45d1-bb7b-5c2ba88d380e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:40.947268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ced438-82e0-45d1-bb7b-5c2ba88d380e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:40.948958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ced438-82e0-45d1-bb7b-5c2ba88d380e service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:40.975496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1113/2206] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:41.993711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21caae-063b-47cb-838f-e43c5a0ca419 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:41.996141 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6e21caae-063b-47cb-838f-e43c5a0ca419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:42.033418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21caae-063b-47cb-838f-e43c5a0ca419 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:42.042989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:42.043804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:42.044254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:42.044699 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] PATH_INFO: `/v3/projects/5002673965844cb887af41eccd644e8c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:42.046024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5002673965844cb887af41eccd644e8c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:42.048230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:42.153965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5002673965844cb887af41eccd644e8c', 'request_id': 'req-1af123e9-e4ec-4e91-b395-6119a72387bd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:42.163589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5002673965844cb887af41eccd644e8c', 'request_id': 'req-1af123e9-e4ec-4e91-b395-6119a72387bd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:42.172610 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5002673965844cb887af41eccd644e8c', 'request_id': 'req-1af123e9-e4ec-4e91-b395-6119a72387bd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:42.254350 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] The token cache is being invalidate because project 5002673965844cb887af41eccd644e8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:07:42.255800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:42.256795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:42.258960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1094/2207] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:07:41 2022] DELETE /identity/v3/projects/5002673965844cb887af41eccd644e8c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:42.262945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ced438-82e0-45d1-bb7b-5c2ba88d380e service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:42.276964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ced438-82e0-45d1-bb7b-5c2ba88d380e service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:42.316838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:42.384334 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb82cff-1677-4e8f-9a90-e5d4fcb8bcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:42.385503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb82cff-1677-4e8f-9a90-e5d4fcb8bcbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:42.385940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb82cff-1677-4e8f-9a90-e5d4fcb8bcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:42.386552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb82cff-1677-4e8f-9a90-e5d4fcb8bcbb admin admin] PATH_INFO: `/v3/users/63748e65c9084f96ad42664be2ea936f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:42.388155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb82cff-1677-4e8f-9a90-e5d4fcb8bcbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=63748e65c9084f96ad42664be2ea936f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:42.391035 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb82cff-1677-4e8f-9a90-e5d4fcb8bcbb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:42.606160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3eb82cff-1677-4e8f-9a90-e5d4fcb8bcbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63748e65c9084f96ad42664be2ea936f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:42.618926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3eb82cff-1677-4e8f-9a90-e5d4fcb8bcbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63748e65c9084f96ad42664be2ea936f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:42.628668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3eb82cff-1677-4e8f-9a90-e5d4fcb8bcbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63748e65c9084f96ad42664be2ea936f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:42.692352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3eb82cff-1677-4e8f-9a90-e5d4fcb8bcbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63748e65c9084f96ad42664be2ea936f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:42.694278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3eb82cff-1677-4e8f-9a90-e5d4fcb8bcbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63748e65c9084f96ad42664be2ea936f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:42.701479 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1114/2208] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:07:42 2022] DELETE /identity/v3/users/63748e65c9084f96ad42664be2ea936f => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:43.639668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:43.641170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:43.655385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af123e9-e4ec-4e91-b395-6119a72387bd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:43.744061 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:43.744704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:43.744983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:43.745579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] PATH_INFO: `/v3/projects/f501df73194b47b3b5479f651eee3832` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:43.747060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f501df73194b47b3b5479f651eee3832)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:43.749062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:43.875886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:43.893000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f501df73194b47b3b5479f651eee3832', 'request_id': 'req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:43.900192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f501df73194b47b3b5479f651eee3832', 'request_id': 'req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:43.907342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f501df73194b47b3b5479f651eee3832', 'request_id': 'req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:44.031569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] The token cache is being invalidate because project f501df73194b47b3b5479f651eee3832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:07:44.031860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:44.032762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5f8f3aaa-4b69-4dc8-88da-b1959ce9e3d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:44.034363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1095/2209] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:07:43 2022] DELETE /identity/v3/projects/f501df73194b47b3b5479f651eee3832 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:44.039310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e3d19-e414-45d4-a0fd-ce0770e7afe0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:44.039814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e3d19-e414-45d4-a0fd-ce0770e7afe0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:44.039922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e3d19-e414-45d4-a0fd-ce0770e7afe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:44.076205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5a0e3d19-e414-45d4-a0fd-ce0770e7afe0 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:44.077667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5a0e3d19-e414-45d4-a0fd-ce0770e7afe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:44.167309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1115/2210] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:44.179910 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5682c8-0026-4a07-9b2c-522685f0558f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:44.180203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5682c8-0026-4a07-9b2c-522685f0558f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:44.180482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5682c8-0026-4a07-9b2c-522685f0558f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:44.207094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-eb5682c8-0026-4a07-9b2c-522685f0558f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:44.208663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eb5682c8-0026-4a07-9b2c-522685f0558f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:44.220373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1096/2211] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:44.221570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e3d19-e414-45d4-a0fd-ce0770e7afe0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:44.232486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e3d19-e414-45d4-a0fd-ce0770e7afe0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:44.242060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18de701-d670-4bfd-a618-cdceb96c089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:44.242940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18de701-d670-4bfd-a618-cdceb96c089e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:44.243330 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18de701-d670-4bfd-a618-cdceb96c089e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:44.243708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18de701-d670-4bfd-a618-cdceb96c089e admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:44.244447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18de701-d670-4bfd-a618-cdceb96c089e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:07:44.244868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18de701-d670-4bfd-a618-cdceb96c089e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:44.246578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18de701-d670-4bfd-a618-cdceb96c089e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:44.261668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1116/2212] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:07:44.265937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5682c8-0026-4a07-9b2c-522685f0558f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:44.277176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5682c8-0026-4a07-9b2c-522685f0558f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:44.284870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4af397-ba4d-4896-bf46-5539a3bf0916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:44.286206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4af397-ba4d-4896-bf46-5539a3bf0916 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:44.286606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4af397-ba4d-4896-bf46-5539a3bf0916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:44.287070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4af397-ba4d-4896-bf46-5539a3bf0916 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:44.287707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4af397-ba4d-4896-bf46-5539a3bf0916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:44.290484 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4af397-ba4d-4896-bf46-5539a3bf0916 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:44.415270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1097/2213] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:07:44 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:07:44.420084 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18de701-d670-4bfd-a618-cdceb96c089e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:44.432618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18de701-d670-4bfd-a618-cdceb96c089e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:44.461316 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:44.494993 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b764f-134b-4650-8c17-f66e43431762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:44.495611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b764f-134b-4650-8c17-f66e43431762 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:44.495915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b764f-134b-4650-8c17-f66e43431762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:44.496172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b764f-134b-4650-8c17-f66e43431762 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:44.496822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b764f-134b-4650-8c17-f66e43431762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:44.498755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b764f-134b-4650-8c17-f66e43431762 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:44.609243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1117/2214] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:07:44 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:07:44.613949 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4af397-ba4d-4896-bf46-5539a3bf0916 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:44.624388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4af397-ba4d-4896-bf46-5539a3bf0916 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:44.629085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf9145-23c5-4467-85c1-4b08cb9e6909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:44.629684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf9145-23c5-4467-85c1-4b08cb9e6909 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:44.629963 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf9145-23c5-4467-85c1-4b08cb9e6909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:44.630234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf9145-23c5-4467-85c1-4b08cb9e6909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:44.630932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf9145-23c5-4467-85c1-4b08cb9e6909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:44.632532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf9145-23c5-4467-85c1-4b08cb9e6909 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:44.643227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1098/2215] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:07:44.647284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b764f-134b-4650-8c17-f66e43431762 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:44.660827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b764f-134b-4650-8c17-f66e43431762 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:44.666214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75f391-0e72-4a64-94b9-7256f05b3f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:44.671256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75f391-0e72-4a64-94b9-7256f05b3f1e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:44.671256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75f391-0e72-4a64-94b9-7256f05b3f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:44.671256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75f391-0e72-4a64-94b9-7256f05b3f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:44.671256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75f391-0e72-4a64-94b9-7256f05b3f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:44.671256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75f391-0e72-4a64-94b9-7256f05b3f1e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:44.682713 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1118/2216] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:07:44.686914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf9145-23c5-4467-85c1-4b08cb9e6909 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:44.697961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf9145-23c5-4467-85c1-4b08cb9e6909 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:44.703378 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d599c-a5c9-4974-8cbc-733b2e480123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:44.704049 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d599c-a5c9-4974-8cbc-733b2e480123 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:44.704332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d599c-a5c9-4974-8cbc-733b2e480123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:44.704610 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d599c-a5c9-4974-8cbc-733b2e480123 admin admin] PATH_INFO: `/v3/projects/3859cdcf8b4c4cb0b99a39c1827ef480/users/7c18d2a86c8a4c81a61c50e222b7e114/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:44.723689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d599c-a5c9-4974-8cbc-733b2e480123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3859cdcf8b4c4cb0b99a39c1827ef480, user_id=7c18d2a86c8a4c81a61c50e222b7e114, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:44.726611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d599c-a5c9-4974-8cbc-733b2e480123 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:44.823915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1099/2217] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:07:44 2022] PUT /identity/v3/projects/3859cdcf8b4c4cb0b99a39c1827ef480/users/7c18d2a86c8a4c81a61c50e222b7e114/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:44.833666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bbfde-0a5f-4cfa-ae18-d36674bef9e3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:44.833941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bbfde-0a5f-4cfa-ae18-d36674bef9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:44.834216 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bbfde-0a5f-4cfa-ae18-d36674bef9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:44.863716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-199bbfde-0a5f-4cfa-ae18-d36674bef9e3 None None] MFA Rules not processed for user `7c18d2a86c8a4c81a61c50e222b7e114`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:44.864930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-199bbfde-0a5f-4cfa-ae18-d36674bef9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:44.930310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1119/2218] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:44.934388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d599c-a5c9-4974-8cbc-733b2e480123 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:45.006200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d599c-a5c9-4974-8cbc-733b2e480123 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:45.013583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4c69b-e981-4fbe-bdc3-11a119567ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:45.013583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4c69b-e981-4fbe-bdc3-11a119567ffd admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:45.013583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4c69b-e981-4fbe-bdc3-11a119567ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:45.013583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4c69b-e981-4fbe-bdc3-11a119567ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:45.013990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4c69b-e981-4fbe-bdc3-11a119567ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:45.014891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4c69b-e981-4fbe-bdc3-11a119567ffd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:45.104785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1100/2219] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:07:44 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:07:45.106237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bbfde-0a5f-4cfa-ae18-d36674bef9e3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:45.117318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bbfde-0a5f-4cfa-ae18-d36674bef9e3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:45.189566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cab3bb-7ed3-4954-adac-e112cbbf11af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:45.190119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab3bb-7ed3-4954-adac-e112cbbf11af admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:45.190403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab3bb-7ed3-4954-adac-e112cbbf11af admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:45.190751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab3bb-7ed3-4954-adac-e112cbbf11af admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:45.191429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cab3bb-7ed3-4954-adac-e112cbbf11af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:45.193367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cab3bb-7ed3-4954-adac-e112cbbf11af admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:45.302808 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1120/2220] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:07:45 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:07:45.306873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4c69b-e981-4fbe-bdc3-11a119567ffd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:45.317388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4c69b-e981-4fbe-bdc3-11a119567ffd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:45.322301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb23121-1b6b-44f4-9c06-2521362bd6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:45.322615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb23121-1b6b-44f4-9c06-2521362bd6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:45.322951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb23121-1b6b-44f4-9c06-2521362bd6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:45.323205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb23121-1b6b-44f4-9c06-2521362bd6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:45.324035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb23121-1b6b-44f4-9c06-2521362bd6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:45.325461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb23121-1b6b-44f4-9c06-2521362bd6e4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:45.337053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1101/2221] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:07:45.341602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cab3bb-7ed3-4954-adac-e112cbbf11af admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:45.354617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cab3bb-7ed3-4954-adac-e112cbbf11af admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:45.360685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f31b28-08af-4615-9bb6-778e5ef333f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:45.361474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f31b28-08af-4615-9bb6-778e5ef333f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:45.361845 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f31b28-08af-4615-9bb6-778e5ef333f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:45.362541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f31b28-08af-4615-9bb6-778e5ef333f6 admin admin] PATH_INFO: `/v3/projects/656ee80332e244bcb1636a8776baa7a4/users/d883559d01ba49ada4a7e8a1230d81cf/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:45.381641 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f31b28-08af-4615-9bb6-778e5ef333f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=656ee80332e244bcb1636a8776baa7a4, user_id=d883559d01ba49ada4a7e8a1230d81cf, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:45.385033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f31b28-08af-4615-9bb6-778e5ef333f6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:45.459302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1121/2222] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:07:45 2022] PUT /identity/v3/projects/656ee80332e244bcb1636a8776baa7a4/users/d883559d01ba49ada4a7e8a1230d81cf/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:45.463217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb23121-1b6b-44f4-9c06-2521362bd6e4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:45.474401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb23121-1b6b-44f4-9c06-2521362bd6e4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:45.498154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:45.545855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d54b0e-8864-49a1-a947-9c46b0780f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:45.546461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d54b0e-8864-49a1-a947-9c46b0780f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:45.546796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d54b0e-8864-49a1-a947-9c46b0780f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:45.547051 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d54b0e-8864-49a1-a947-9c46b0780f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:45.547692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d54b0e-8864-49a1-a947-9c46b0780f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:45.549277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d54b0e-8864-49a1-a947-9c46b0780f93 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:45.559634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1102/2223] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:07:45.563255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f31b28-08af-4615-9bb6-778e5ef333f6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:45.569119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:45.574338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f31b28-08af-4615-9bb6-778e5ef333f6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:45.579075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669446f-03ba-4b7f-8387-9c4ca59a10bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:45.579719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669446f-03ba-4b7f-8387-9c4ca59a10bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:45.580020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669446f-03ba-4b7f-8387-9c4ca59a10bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:45.580294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669446f-03ba-4b7f-8387-9c4ca59a10bb admin admin] PATH_INFO: `/v3/projects/656ee80332e244bcb1636a8776baa7a4/users/d883559d01ba49ada4a7e8a1230d81cf/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:45.582737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669446f-03ba-4b7f-8387-9c4ca59a10bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=656ee80332e244bcb1636a8776baa7a4, user_id=d883559d01ba49ada4a7e8a1230d81cf, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:45.585491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669446f-03ba-4b7f-8387-9c4ca59a10bb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:45.595820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1122/2224] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:07:45 2022] PUT /identity/v3/projects/656ee80332e244bcb1636a8776baa7a4/users/d883559d01ba49ada4a7e8a1230d81cf/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:45.607018 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf6c5d-14e2-43df-b87e-a197b1e4b490 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:45.607305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf6c5d-14e2-43df-b87e-a197b1e4b490 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:45.607578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf6c5d-14e2-43df-b87e-a197b1e4b490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:45.635000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9faf6c5d-14e2-43df-b87e-a197b1e4b490 None None] MFA Rules not processed for user `d883559d01ba49ada4a7e8a1230d81cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:45.636218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9faf6c5d-14e2-43df-b87e-a197b1e4b490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:45.703753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1103/2225] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:45.715067 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669446f-03ba-4b7f-8387-9c4ca59a10bb admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:45.716758 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0669446f-03ba-4b7f-8387-9c4ca59a10bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:45.777510 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669446f-03ba-4b7f-8387-9c4ca59a10bb admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:45.781217 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe03e6-16da-45d0-b21c-d28103a55bbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:45.781825 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe03e6-16da-45d0-b21c-d28103a55bbf service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:45.782364 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe03e6-16da-45d0-b21c-d28103a55bbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:45.782637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe03e6-16da-45d0-b21c-d28103a55bbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:45.792189 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe03e6-16da-45d0-b21c-d28103a55bbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:45.793604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe03e6-16da-45d0-b21c-d28103a55bbf service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:45.803404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1123/2226] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:45.867922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf6c5d-14e2-43df-b87e-a197b1e4b490 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:45.928932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf6c5d-14e2-43df-b87e-a197b1e4b490 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:45.933430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:45.934044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:45.934319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:45.934587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] PATH_INFO: `/v3/users/7c18d2a86c8a4c81a61c50e222b7e114` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:45.935724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c18d2a86c8a4c81a61c50e222b7e114)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:45.937634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:46.108108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c18d2a86c8a4c81a61c50e222b7e114'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:46.109301 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:46.121391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c18d2a86c8a4c81a61c50e222b7e114'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:46.128529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c18d2a86c8a4c81a61c50e222b7e114'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:46.190713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c18d2a86c8a4c81a61c50e222b7e114'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:46.195246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c18d2a86c8a4c81a61c50e222b7e114'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:46.198629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1104/2227] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:07:45 2022] DELETE /identity/v3/users/7c18d2a86c8a4c81a61c50e222b7e114 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:46.203517 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a917a-7477-4572-9dae-a3105c01402e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:46.203937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a917a-7477-4572-9dae-a3105c01402e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:46.204093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a917a-7477-4572-9dae-a3105c01402e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:46.226645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f21a917a-7477-4572-9dae-a3105c01402e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:46.227962 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f21a917a-7477-4572-9dae-a3105c01402e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:46.293619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1124/2228] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:46.303532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:46.304971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:46.364212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaed91c-1e0a-48d3-a17a-5ec6a485a065 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:46.367717 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5376a1-91c9-4932-8347-82da32f70a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:46.368340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5376a1-91c9-4932-8347-82da32f70a99 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:46.368759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5376a1-91c9-4932-8347-82da32f70a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:46.369042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5376a1-91c9-4932-8347-82da32f70a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:46.379881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5376a1-91c9-4932-8347-82da32f70a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:46.381321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5376a1-91c9-4932-8347-82da32f70a99 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:46.391299 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1105/2229] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:46.918267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a917a-7477-4572-9dae-a3105c01402e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:46.922609 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f21a917a-7477-4572-9dae-a3105c01402e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:46.932486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a917a-7477-4572-9dae-a3105c01402e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:46.937003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:46.937626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:46.938109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:46.939117 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] PATH_INFO: `/v3/projects/3859cdcf8b4c4cb0b99a39c1827ef480` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:46.939920 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3859cdcf8b4c4cb0b99a39c1827ef480)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:46.942743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:47.053463 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3859cdcf8b4c4cb0b99a39c1827ef480', 'request_id': 'req-2575d5c6-977e-403d-b4ad-96321983843d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:47.063690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3859cdcf8b4c4cb0b99a39c1827ef480', 'request_id': 'req-2575d5c6-977e-403d-b4ad-96321983843d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:47.071399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3859cdcf8b4c4cb0b99a39c1827ef480', 'request_id': 'req-2575d5c6-977e-403d-b4ad-96321983843d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:47.186628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] The token cache is being invalidate because project 3859cdcf8b4c4cb0b99a39c1827ef480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:07:47.186957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:47.187622 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:47.189401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1125/2230] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:07:46 2022] DELETE /identity/v3/projects/3859cdcf8b4c4cb0b99a39c1827ef480 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:47.195420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5376a1-91c9-4932-8347-82da32f70a99 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:47.206431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5376a1-91c9-4932-8347-82da32f70a99 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:47.276487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cebd5-cb19-4a09-a9b0-fc3330c13270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:47.277194 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cebd5-cb19-4a09-a9b0-fc3330c13270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:47.278119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cebd5-cb19-4a09-a9b0-fc3330c13270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:47.278424 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cebd5-cb19-4a09-a9b0-fc3330c13270 admin admin] PATH_INFO: `/v3/users/d883559d01ba49ada4a7e8a1230d81cf` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:47.279836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cebd5-cb19-4a09-a9b0-fc3330c13270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d883559d01ba49ada4a7e8a1230d81cf)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:47.282542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cebd5-cb19-4a09-a9b0-fc3330c13270 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:47.521210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bf8cebd5-cb19-4a09-a9b0-fc3330c13270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd883559d01ba49ada4a7e8a1230d81cf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:47.534514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bf8cebd5-cb19-4a09-a9b0-fc3330c13270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd883559d01ba49ada4a7e8a1230d81cf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:47.539443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:47.542936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bf8cebd5-cb19-4a09-a9b0-fc3330c13270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd883559d01ba49ada4a7e8a1230d81cf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:47.609098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bf8cebd5-cb19-4a09-a9b0-fc3330c13270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd883559d01ba49ada4a7e8a1230d81cf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:47.610915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bf8cebd5-cb19-4a09-a9b0-fc3330c13270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd883559d01ba49ada4a7e8a1230d81cf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:47.617951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1106/2231] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:07:47 2022] DELETE /identity/v3/users/d883559d01ba49ada4a7e8a1230d81cf => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:48.247143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:48.248694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:48.262229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575d5c6-977e-403d-b4ad-96321983843d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:48.277535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:48.346370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:48.347083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:48.347364 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:48.348170 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] PATH_INFO: `/v3/projects/656ee80332e244bcb1636a8776baa7a4` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:48.349352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=656ee80332e244bcb1636a8776baa7a4)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:48.353053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:48.528357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '656ee80332e244bcb1636a8776baa7a4', 'request_id': 'req-7b77e0c9-c601-4ac1-8750-0e423e6cef89'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:48.542886 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '656ee80332e244bcb1636a8776baa7a4', 'request_id': 'req-7b77e0c9-c601-4ac1-8750-0e423e6cef89'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:48.548697 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '656ee80332e244bcb1636a8776baa7a4', 'request_id': 'req-7b77e0c9-c601-4ac1-8750-0e423e6cef89'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:48.681804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] The token cache is being invalidate because project 656ee80332e244bcb1636a8776baa7a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:07:48.682331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:48.683150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7b77e0c9-c601-4ac1-8750-0e423e6cef89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:48.685023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1126/2232] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:07:48 2022] DELETE /identity/v3/projects/656ee80332e244bcb1636a8776baa7a4 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:48.689975 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9211-1cfb-4313-aa1b-3f0f26e94fdd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:48.690505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9211-1cfb-4313-aa1b-3f0f26e94fdd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:48.690505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9211-1cfb-4313-aa1b-3f0f26e94fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:48.712440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a64d9211-1cfb-4313-aa1b-3f0f26e94fdd None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:48.714185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a64d9211-1cfb-4313-aa1b-3f0f26e94fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:48.783153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1107/2233] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:48.791765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b9306-7f1d-4945-bd78-2e5125c50656 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:48.792056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b9306-7f1d-4945-bd78-2e5125c50656 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:48.792336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b9306-7f1d-4945-bd78-2e5125c50656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:48.813388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-015b9306-7f1d-4945-bd78-2e5125c50656 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:48.814604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-015b9306-7f1d-4945-bd78-2e5125c50656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:48.827976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1127/2234] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:07:48.829321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d9211-1cfb-4313-aa1b-3f0f26e94fdd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:48.844691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d9211-1cfb-4313-aa1b-3f0f26e94fdd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:48.857286 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf9ff8-da0c-446a-b2c9-333fabf1f420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:48.858276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9ff8-da0c-446a-b2c9-333fabf1f420 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:48.858630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9ff8-da0c-446a-b2c9-333fabf1f420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:48.858955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf9ff8-da0c-446a-b2c9-333fabf1f420 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:48.860645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf9ff8-da0c-446a-b2c9-333fabf1f420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:07:48.861058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf9ff8-da0c-446a-b2c9-333fabf1f420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:48.863057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf9ff8-da0c-446a-b2c9-333fabf1f420 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:48.881010 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1108/2235] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:07:48.887213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b9306-7f1d-4945-bd78-2e5125c50656 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:48.911693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b9306-7f1d-4945-bd78-2e5125c50656 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:48.933097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb348b-22ec-4433-b9ef-6fcd4461ea05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:48.933605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb348b-22ec-4433-b9ef-6fcd4461ea05 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:48.933991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb348b-22ec-4433-b9ef-6fcd4461ea05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:48.934876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb348b-22ec-4433-b9ef-6fcd4461ea05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:48.941834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb348b-22ec-4433-b9ef-6fcd4461ea05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:48.943504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb348b-22ec-4433-b9ef-6fcd4461ea05 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:49.084201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1128/2236] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:07:48 2022] POST /identity/v3/projects => generated 409 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:07:49.089310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf9ff8-da0c-446a-b2c9-333fabf1f420 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:49.102007 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf9ff8-da0c-446a-b2c9-333fabf1f420 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:49.189468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc537690-2561-4140-b29e-bca07f198c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:49.194867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc537690-2561-4140-b29e-bca07f198c95 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:49.194867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc537690-2561-4140-b29e-bca07f198c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:49.194867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc537690-2561-4140-b29e-bca07f198c95 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:49.194867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc537690-2561-4140-b29e-bca07f198c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:49.194867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc537690-2561-4140-b29e-bca07f198c95 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:49.301638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1109/2237] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:07:49 2022] POST /identity/v3/users => generated 305 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:07:49.302742 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb348b-22ec-4433-b9ef-6fcd4461ea05 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:49.314020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb348b-22ec-4433-b9ef-6fcd4461ea05 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:49.319410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333f324-6bc7-4d85-98d5-15292ac875de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:49.320022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333f324-6bc7-4d85-98d5-15292ac875de admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:49.320411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333f324-6bc7-4d85-98d5-15292ac875de admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:49.320631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333f324-6bc7-4d85-98d5-15292ac875de admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:49.321281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333f324-6bc7-4d85-98d5-15292ac875de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:49.322931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333f324-6bc7-4d85-98d5-15292ac875de admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:49.338015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1129/2238] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:07:49.341974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc537690-2561-4140-b29e-bca07f198c95 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:49.353744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc537690-2561-4140-b29e-bca07f198c95 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:49.358882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defc921-03fb-432e-ab9d-2d7ab2e0c367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:49.359503 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defc921-03fb-432e-ab9d-2d7ab2e0c367 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:49.359784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defc921-03fb-432e-ab9d-2d7ab2e0c367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:49.360056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defc921-03fb-432e-ab9d-2d7ab2e0c367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:49.360710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defc921-03fb-432e-ab9d-2d7ab2e0c367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:49.362360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defc921-03fb-432e-ab9d-2d7ab2e0c367 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:49.375113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1110/2239] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:07:49.379789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333f324-6bc7-4d85-98d5-15292ac875de admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:49.391052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333f324-6bc7-4d85-98d5-15292ac875de admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:49.396451 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f82164-7f84-458d-972b-8ab7ecafc473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:49.397018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f82164-7f84-458d-972b-8ab7ecafc473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:49.397298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f82164-7f84-458d-972b-8ab7ecafc473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:49.397571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f82164-7f84-458d-972b-8ab7ecafc473 admin admin] PATH_INFO: `/v3/projects/06a88fa4fff8413b997fd3b0603e9d42/users/40933239407a4eab9a0ff7f94d51a978/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:49.420789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f82164-7f84-458d-972b-8ab7ecafc473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a88fa4fff8413b997fd3b0603e9d42, user_id=40933239407a4eab9a0ff7f94d51a978, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:49.423531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f82164-7f84-458d-972b-8ab7ecafc473 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:49.496668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1130/2240] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:07:49 2022] PUT /identity/v3/projects/06a88fa4fff8413b997fd3b0603e9d42/users/40933239407a4eab9a0ff7f94d51a978/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:07:49.507397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45fe30-71f6-43cf-8e6d-58518d3ec6a9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:49.507674 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45fe30-71f6-43cf-8e6d-58518d3ec6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:49.507943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45fe30-71f6-43cf-8e6d-58518d3ec6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:49.535808 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fa45fe30-71f6-43cf-8e6d-58518d3ec6a9 None None] MFA Rules not processed for user `40933239407a4eab9a0ff7f94d51a978`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:49.537041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fa45fe30-71f6-43cf-8e6d-58518d3ec6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:49.580907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:49.604139 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1111/2241] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:49 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:49.609106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f82164-7f84-458d-972b-8ab7ecafc473 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:49.657611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:49.675224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f82164-7f84-458d-972b-8ab7ecafc473 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:49.681824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce07df-be0e-44a2-acd1-1cc7dc649085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:49.681824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce07df-be0e-44a2-acd1-1cc7dc649085 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:49.681824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce07df-be0e-44a2-acd1-1cc7dc649085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:49.681824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce07df-be0e-44a2-acd1-1cc7dc649085 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:49.682235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce07df-be0e-44a2-acd1-1cc7dc649085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:49.684278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce07df-be0e-44a2-acd1-1cc7dc649085 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:49.707420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45fe30-71f6-43cf-8e6d-58518d3ec6a9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:49.708834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fa45fe30-71f6-43cf-8e6d-58518d3ec6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:49.729600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45fe30-71f6-43cf-8e6d-58518d3ec6a9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:49.734241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9b3fa-ae37-4608-b2bc-9726c4a81362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:49.734926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9b3fa-ae37-4608-b2bc-9726c4a81362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:49.739397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9b3fa-ae37-4608-b2bc-9726c4a81362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:49.739397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9b3fa-ae37-4608-b2bc-9726c4a81362 admin admin] PATH_INFO: `/v3/users/fa542d7bf6134efb99f25456d852f600` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:49.739397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9b3fa-ae37-4608-b2bc-9726c4a81362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa542d7bf6134efb99f25456d852f600)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:49.739397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9b3fa-ae37-4608-b2bc-9726c4a81362 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:49.798145 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1131/2242] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:07:49 2022] POST /identity/v3/projects => generated 409 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:07:49.802514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce07df-be0e-44a2-acd1-1cc7dc649085 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:49.867452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce07df-be0e-44a2-acd1-1cc7dc649085 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:49.872138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f91c78-c318-48c7-b8b7-0549b96c0533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:49.872749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f91c78-c318-48c7-b8b7-0549b96c0533 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:49.873030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f91c78-c318-48c7-b8b7-0549b96c0533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:49.873302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f91c78-c318-48c7-b8b7-0549b96c0533 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:49.873948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f91c78-c318-48c7-b8b7-0549b96c0533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:49.875966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f91c78-c318-48c7-b8b7-0549b96c0533 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:49.940323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-afa9b3fa-ae37-4608-b2bc-9726c4a81362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa542d7bf6134efb99f25456d852f600'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:49.952009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1132/2243] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:07:49 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:07:49.955610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f91c78-c318-48c7-b8b7-0549b96c0533 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:49.974083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f91c78-c318-48c7-b8b7-0549b96c0533 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:49.974594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-afa9b3fa-ae37-4608-b2bc-9726c4a81362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa542d7bf6134efb99f25456d852f600'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:49.982229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-afa9b3fa-ae37-4608-b2bc-9726c4a81362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa542d7bf6134efb99f25456d852f600'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:49.982733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db8b4d-7db7-4282-90f8-3ba6597eb313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:49.984095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db8b4d-7db7-4282-90f8-3ba6597eb313 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:49.984650 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db8b4d-7db7-4282-90f8-3ba6597eb313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:49.986014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db8b4d-7db7-4282-90f8-3ba6597eb313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:49.987437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db8b4d-7db7-4282-90f8-3ba6597eb313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:49.989842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db8b4d-7db7-4282-90f8-3ba6597eb313 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:50.002918 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1133/2244] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:07:50.007197 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db8b4d-7db7-4282-90f8-3ba6597eb313 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:50.017033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db8b4d-7db7-4282-90f8-3ba6597eb313 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:50.021792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fbd40-a6ef-4fe1-aa54-80c698acdddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:50.022418 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fbd40-a6ef-4fe1-aa54-80c698acdddc admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:50.022732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fbd40-a6ef-4fe1-aa54-80c698acdddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:50.023006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fbd40-a6ef-4fe1-aa54-80c698acdddc admin admin] PATH_INFO: `/v3/projects/0c295a9da79b4eb695dd84f6ae3559b0/users/6f99dc80df784901892c91c8e89fdda4/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:50.038410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fbd40-a6ef-4fe1-aa54-80c698acdddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c295a9da79b4eb695dd84f6ae3559b0, user_id=6f99dc80df784901892c91c8e89fdda4, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:50.041138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fbd40-a6ef-4fe1-aa54-80c698acdddc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:50.057327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-afa9b3fa-ae37-4608-b2bc-9726c4a81362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa542d7bf6134efb99f25456d852f600'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:50.057327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-afa9b3fa-ae37-4608-b2bc-9726c4a81362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa542d7bf6134efb99f25456d852f600'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:50.061029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1112/2245] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:07:49 2022] DELETE /identity/v3/users/fa542d7bf6134efb99f25456d852f600 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:07:50.065597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf32fe7-a147-475f-a7b8-6ca8c4230c00 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:50.065889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf32fe7-a147-475f-a7b8-6ca8c4230c00 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:50.066170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf32fe7-a147-475f-a7b8-6ca8c4230c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:50.084753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dbf32fe7-a147-475f-a7b8-6ca8c4230c00 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:50.085965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dbf32fe7-a147-475f-a7b8-6ca8c4230c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:50.140552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1134/2246] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:07:50 2022] PUT /identity/v3/projects/0c295a9da79b4eb695dd84f6ae3559b0/users/6f99dc80df784901892c91c8e89fdda4/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:07:50.144858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fbd40-a6ef-4fe1-aa54-80c698acdddc admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:50.149742 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-806fbd40-a6ef-4fe1-aa54-80c698acdddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:50.149886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1113/2247] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:07:50.157168 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf32fe7-a147-475f-a7b8-6ca8c4230c00 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:50.159044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dbf32fe7-a147-475f-a7b8-6ca8c4230c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:50.237579 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fbd40-a6ef-4fe1-aa54-80c698acdddc admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:50.244885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2097b-deaa-4d7d-9e64-939f4f0c0571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:50.245526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2097b-deaa-4d7d-9e64-939f4f0c0571 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:50.250438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2097b-deaa-4d7d-9e64-939f4f0c0571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:50.250438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2097b-deaa-4d7d-9e64-939f4f0c0571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:50.250438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2097b-deaa-4d7d-9e64-939f4f0c0571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:50.250438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2097b-deaa-4d7d-9e64-939f4f0c0571 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:50.250797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf32fe7-a147-475f-a7b8-6ca8c4230c00 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:50.263111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1135/2248] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:07:50.264392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:50.266548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2097b-deaa-4d7d-9e64-939f4f0c0571 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:50.278110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2097b-deaa-4d7d-9e64-939f4f0c0571 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:50.283871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01c366-1834-4e3e-8590-761ed13ce2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:50.284588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01c366-1834-4e3e-8590-761ed13ce2c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:50.284963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01c366-1834-4e3e-8590-761ed13ce2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:50.285841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01c366-1834-4e3e-8590-761ed13ce2c9 admin admin] PATH_INFO: `/v3/projects/0c295a9da79b4eb695dd84f6ae3559b0/users/6f99dc80df784901892c91c8e89fdda4/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:50.288072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01c366-1834-4e3e-8590-761ed13ce2c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c295a9da79b4eb695dd84f6ae3559b0, user_id=6f99dc80df784901892c91c8e89fdda4, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:50.293123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01c366-1834-4e3e-8590-761ed13ce2c9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:50.302932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1136/2249] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:07:50 2022] PUT /identity/v3/projects/0c295a9da79b4eb695dd84f6ae3559b0/users/6f99dc80df784901892c91c8e89fdda4/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:50.314710 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f22e2-3b42-4e10-afd7-17a34986c835 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:50.314955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f22e2-3b42-4e10-afd7-17a34986c835 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:50.315239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f22e2-3b42-4e10-afd7-17a34986c835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:50.325404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:50.325633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69f6ea-0553-4211-9058-f3e001546678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:50.326054 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f6ea-0553-4211-9058-f3e001546678 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:50.326338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f6ea-0553-4211-9058-f3e001546678 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:50.326612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f6ea-0553-4211-9058-f3e001546678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:50.345841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69f6ea-0553-4211-9058-f3e001546678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:50.347361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-743f22e2-3b42-4e10-afd7-17a34986c835 None None] MFA Rules not processed for user `6f99dc80df784901892c91c8e89fdda4`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:50.347610 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69f6ea-0553-4211-9058-f3e001546678 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:50.349399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-743f22e2-3b42-4e10-afd7-17a34986c835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:50.362902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1114/2250] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:50.429153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1137/2251] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:50 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:07:50.436283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69f6ea-0553-4211-9058-f3e001546678 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:50.441278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6a69f6ea-0553-4211-9058-f3e001546678 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:50.517016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69f6ea-0553-4211-9058-f3e001546678 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:50.576253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c23ed-b18a-488c-b52f-3f02c505583f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:50.576253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c23ed-b18a-488c-b52f-3f02c505583f service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:50.576253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c23ed-b18a-488c-b52f-3f02c505583f service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:50.576253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c23ed-b18a-488c-b52f-3f02c505583f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:50.579744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c23ed-b18a-488c-b52f-3f02c505583f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:50.581217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c23ed-b18a-488c-b52f-3f02c505583f service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:50.591585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1115/2252] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:50 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:51.029884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f22e2-3b42-4e10-afd7-17a34986c835 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:51.040179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f22e2-3b42-4e10-afd7-17a34986c835 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:51.044015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6b85f-4f3c-4f46-8f57-016126993c81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:51.044631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6b85f-4f3c-4f46-8f57-016126993c81 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:51.044916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6b85f-4f3c-4f46-8f57-016126993c81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:51.045198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6b85f-4f3c-4f46-8f57-016126993c81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:51.047059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f8c6b85f-4f3c-4f46-8f57-016126993c81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:51.074390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c23ed-b18a-488c-b52f-3f02c505583f service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:51.075758 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1a2c23ed-b18a-488c-b52f-3f02c505583f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:51.125627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6b85f-4f3c-4f46-8f57-016126993c81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:51.127693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6b85f-4f3c-4f46-8f57-016126993c81 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:51.139798 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1138/2253] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:51 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:51.158829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c23ed-b18a-488c-b52f-3f02c505583f service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:51.163903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:51.164545 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:51.164830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:51.165111 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] PATH_INFO: `/v3/projects/0f74dfa55d37442996d298d4df1d7734` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:51.166244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f74dfa55d37442996d298d4df1d7734)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:51.168217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:51.186514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:51.280274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f74dfa55d37442996d298d4df1d7734', 'request_id': 'req-9adbc98d-cd5d-4661-a043-c96f7876e57b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:51.295652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f74dfa55d37442996d298d4df1d7734', 'request_id': 'req-9adbc98d-cd5d-4661-a043-c96f7876e57b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:51.302558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f74dfa55d37442996d298d4df1d7734', 'request_id': 'req-9adbc98d-cd5d-4661-a043-c96f7876e57b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:51.399942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] The token cache is being invalidate because project 0f74dfa55d37442996d298d4df1d7734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:07:51.400329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:51.400872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9adbc98d-cd5d-4661-a043-c96f7876e57b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:51.402646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1116/2254] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:07:51 2022] DELETE /identity/v3/projects/0f74dfa55d37442996d298d4df1d7734 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:53.939138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6b85f-4f3c-4f46-8f57-016126993c81 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:53.940541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f8c6b85f-4f3c-4f46-8f57-016126993c81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:53.958155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6b85f-4f3c-4f46-8f57-016126993c81 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:54.012399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:54.012971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:54.013246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:54.013517 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] PATH_INFO: `/v3/users/edd09843082748a79d26452687f7a989` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:54.014637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd09843082748a79d26452687f7a989)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:54.016582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:54.166315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:54.180022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd09843082748a79d26452687f7a989'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:54.192801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd09843082748a79d26452687f7a989'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:54.200072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd09843082748a79d26452687f7a989'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:54.282594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd09843082748a79d26452687f7a989'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:54.283497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd09843082748a79d26452687f7a989'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:54.293375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1139/2255] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:07:53 2022] DELETE /identity/v3/users/edd09843082748a79d26452687f7a989 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:54.295576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d17bd-e071-4538-8322-3e89952e8909 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:54.296051 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d17bd-e071-4538-8322-3e89952e8909 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:54.296145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d17bd-e071-4538-8322-3e89952e8909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:54.315319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-177d17bd-e071-4538-8322-3e89952e8909 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:54.316531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-177d17bd-e071-4538-8322-3e89952e8909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:54.375869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1117/2256] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:07:54.385830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:54.387301 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:54.447404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1f3f1-8bb3-43d1-9c23-a337d2fb743b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:54.451044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ae0f6-3e59-49a1-9134-60455ca19f0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:54.451789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ae0f6-3e59-49a1-9134-60455ca19f0a service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:54.452074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ae0f6-3e59-49a1-9134-60455ca19f0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:54.452344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ae0f6-3e59-49a1-9134-60455ca19f0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:54.461621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ae0f6-3e59-49a1-9134-60455ca19f0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:54.463074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ae0f6-3e59-49a1-9134-60455ca19f0a service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:54.472687 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1140/2257] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:07:54.880704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d17bd-e071-4538-8322-3e89952e8909 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:54.884522 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-177d17bd-e071-4538-8322-3e89952e8909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:54.894351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d17bd-e071-4538-8322-3e89952e8909 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:54.899549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:54.900173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:54.900490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:54.900751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] PATH_INFO: `/v3/projects/1a12314f7d5f419bb45e3564656553e3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:54.901872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a12314f7d5f419bb45e3564656553e3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:54.903810 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:55.007320 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a12314f7d5f419bb45e3564656553e3', 'request_id': 'req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:55.013382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a12314f7d5f419bb45e3564656553e3', 'request_id': 'req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:55.018900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a12314f7d5f419bb45e3564656553e3', 'request_id': 'req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:55.107262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] The token cache is being invalidate because project 1a12314f7d5f419bb45e3564656553e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:07:55.107562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:55.108194 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ff9537a0-ab0a-4a0b-9679-6132a1e2e5ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:07:55.109688 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1118/2258] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:07:54 2022] DELETE /identity/v3/projects/1a12314f7d5f419bb45e3564656553e3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:55.114821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d686e-1c56-4b91-a7f8-a9d8eab24c25 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:55.115315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d686e-1c56-4b91-a7f8-a9d8eab24c25 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:55.115422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d686e-1c56-4b91-a7f8-a9d8eab24c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:55.135889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2f1d686e-1c56-4b91-a7f8-a9d8eab24c25 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:55.137104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2f1d686e-1c56-4b91-a7f8-a9d8eab24c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:55.197462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1141/2259] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:55.205298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726dca4-6ca5-4f7c-8479-a6ec79b4b14f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:55.205588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726dca4-6ca5-4f7c-8479-a6ec79b4b14f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:55.205891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726dca4-6ca5-4f7c-8479-a6ec79b4b14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:55.226424 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b726dca4-6ca5-4f7c-8479-a6ec79b4b14f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:55.227698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b726dca4-6ca5-4f7c-8479-a6ec79b4b14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:55.235800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1119/2260] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:55.240081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d686e-1c56-4b91-a7f8-a9d8eab24c25 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:55.250231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d686e-1c56-4b91-a7f8-a9d8eab24c25 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:55.260402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c6044-8ec8-4671-8a61-38dcd3e8f072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:55.260402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c6044-8ec8-4671-8a61-38dcd3e8f072 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:55.260402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c6044-8ec8-4671-8a61-38dcd3e8f072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:55.260402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c6044-8ec8-4671-8a61-38dcd3e8f072 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:55.260774 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c6044-8ec8-4671-8a61-38dcd3e8f072 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:07:55.260774 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c6044-8ec8-4671-8a61-38dcd3e8f072 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:55.262670 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c6044-8ec8-4671-8a61-38dcd3e8f072 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:55.275454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1142/2261] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:07:55.279835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726dca4-6ca5-4f7c-8479-a6ec79b4b14f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:55.289100 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726dca4-6ca5-4f7c-8479-a6ec79b4b14f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:55.293694 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e90c3-737c-4a05-b50f-d1136015acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:55.294284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e90c3-737c-4a05-b50f-d1136015acff admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:55.294559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e90c3-737c-4a05-b50f-d1136015acff admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:55.294862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e90c3-737c-4a05-b50f-d1136015acff admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:55.295542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e90c3-737c-4a05-b50f-d1136015acff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:55.297396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e90c3-737c-4a05-b50f-d1136015acff admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:55.423003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:55.430586 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1120/2262] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:07:55 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:07:55.434897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c6044-8ec8-4671-8a61-38dcd3e8f072 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:55.445541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c6044-8ec8-4671-8a61-38dcd3e8f072 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:55.478095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:07:55.502695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85ee8e-476c-425c-9659-4c9892d97fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:55.503342 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85ee8e-476c-425c-9659-4c9892d97fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:55.503619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85ee8e-476c-425c-9659-4c9892d97fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:55.503889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85ee8e-476c-425c-9659-4c9892d97fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:55.504629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85ee8e-476c-425c-9659-4c9892d97fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:55.507048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85ee8e-476c-425c-9659-4c9892d97fd0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:55.612996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1143/2263] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:07:55 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:07:55.617263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e90c3-737c-4a05-b50f-d1136015acff admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:55.628021 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e90c3-737c-4a05-b50f-d1136015acff admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:55.632779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a99f7-c1aa-4f84-a057-277db6da3904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:55.633388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a99f7-c1aa-4f84-a057-277db6da3904 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:55.633672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a99f7-c1aa-4f84-a057-277db6da3904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:55.633947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a99f7-c1aa-4f84-a057-277db6da3904 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:55.634595 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a99f7-c1aa-4f84-a057-277db6da3904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:55.636317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a99f7-c1aa-4f84-a057-277db6da3904 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:55.646908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1121/2264] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:07:55.651107 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85ee8e-476c-425c-9659-4c9892d97fd0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:55.662126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85ee8e-476c-425c-9659-4c9892d97fd0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:55.667034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a4196-ca61-4896-a4b6-e870077e59e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:55.667998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a4196-ca61-4896-a4b6-e870077e59e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:55.668689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a4196-ca61-4896-a4b6-e870077e59e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:55.669327 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a4196-ca61-4896-a4b6-e870077e59e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:55.670385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a4196-ca61-4896-a4b6-e870077e59e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:55.672186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a4196-ca61-4896-a4b6-e870077e59e2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:55.683047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1144/2265] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:07:55.687944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a99f7-c1aa-4f84-a057-277db6da3904 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:55.699397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a99f7-c1aa-4f84-a057-277db6da3904 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:55.704142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1c8f5-7ef6-4b3c-92d6-96bf43f18e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:55.704795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1c8f5-7ef6-4b3c-92d6-96bf43f18e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:55.705077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1c8f5-7ef6-4b3c-92d6-96bf43f18e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:55.705352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1c8f5-7ef6-4b3c-92d6-96bf43f18e6d admin admin] PATH_INFO: `/v3/projects/794ad972bfa844538f6f71d62e7bd163/users/afbb1d9c9bb34fbd8bfc78710c86d1f5/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:55.720489 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1c8f5-7ef6-4b3c-92d6-96bf43f18e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=794ad972bfa844538f6f71d62e7bd163, user_id=afbb1d9c9bb34fbd8bfc78710c86d1f5, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:55.723261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1c8f5-7ef6-4b3c-92d6-96bf43f18e6d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:55.802173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1122/2266] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:07:55 2022] PUT /identity/v3/projects/794ad972bfa844538f6f71d62e7bd163/users/afbb1d9c9bb34fbd8bfc78710c86d1f5/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:07:55.807124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb015e8-0795-4c6f-9234-6f4ef64d7a2b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:55.807415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb015e8-0795-4c6f-9234-6f4ef64d7a2b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:55.807694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb015e8-0795-4c6f-9234-6f4ef64d7a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:55.826982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-acb015e8-0795-4c6f-9234-6f4ef64d7a2b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:07:55.828192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-acb015e8-0795-4c6f-9234-6f4ef64d7a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:55.886057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1145/2267] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:07:55.896096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1c8f5-7ef6-4b3c-92d6-96bf43f18e6d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:07:55.897775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-39c1c8f5-7ef6-4b3c-92d6-96bf43f18e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:07:55.954561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1c8f5-7ef6-4b3c-92d6-96bf43f18e6d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:07:55.958241 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c103c-3254-4b28-aa2b-e3cc6c90f570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:07:55.958871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c103c-3254-4b28-aa2b-e3cc6c90f570 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:07:55.959154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c103c-3254-4b28-aa2b-e3cc6c90f570 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:07:55.959429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c103c-3254-4b28-aa2b-e3cc6c90f570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:07:55.968213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c103c-3254-4b28-aa2b-e3cc6c90f570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:07:55.969640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c103c-3254-4b28-aa2b-e3cc6c90f570 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:07:55.979374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1123/2268] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:08:01.200257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63218321-90e7-40fe-bd0f-4e6c0f489d52 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:01.200740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63218321-90e7-40fe-bd0f-4e6c0f489d52 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:01.200968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63218321-90e7-40fe-bd0f-4e6c0f489d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:01.229173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-63218321-90e7-40fe-bd0f-4e6c0f489d52 None None] MFA Rules not processed for user `5a85cff936184b5d94375fd103493950`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:01.230891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-63218321-90e7-40fe-bd0f-4e6c0f489d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:01.322843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1146/2269] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:08:01.656069 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c103c-3254-4b28-aa2b-e3cc6c90f570 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:01.657753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-be5c103c-3254-4b28-aa2b-e3cc6c90f570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:01.684185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c103c-3254-4b28-aa2b-e3cc6c90f570 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:01.689589 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:01.690498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:01.691240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:01.692701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] PATH_INFO: `/v3/users/40933239407a4eab9a0ff7f94d51a978` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:01.694159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40933239407a4eab9a0ff7f94d51a978)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:01.696522 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:01.932748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40933239407a4eab9a0ff7f94d51a978'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:01.956634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40933239407a4eab9a0ff7f94d51a978'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:01.976438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40933239407a4eab9a0ff7f94d51a978'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:02.044907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40933239407a4eab9a0ff7f94d51a978'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:02.045882 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40933239407a4eab9a0ff7f94d51a978'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:02.054083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1124/2270] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:08:01 2022] DELETE /identity/v3/users/40933239407a4eab9a0ff7f94d51a978 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:08:02.058695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b3240-2023-4395-9882-7c0a8578d916 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:02.058994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b3240-2023-4395-9882-7c0a8578d916 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:02.059307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b3240-2023-4395-9882-7c0a8578d916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:02.084262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-552b3240-2023-4395-9882-7c0a8578d916 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:02.085576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-552b3240-2023-4395-9882-7c0a8578d916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:02.205243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1147/2271] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:08:02.227428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:02.227428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:02.400437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:02.419527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfc640-b71c-4789-9f0c-9065e28c2133 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:02.427706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10596e78-a226-483c-bef1-172c1bb9d5b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:02.435105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10596e78-a226-483c-bef1-172c1bb9d5b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:02.435530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10596e78-a226-483c-bef1-172c1bb9d5b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:02.435901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10596e78-a226-483c-bef1-172c1bb9d5b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:02.448616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10596e78-a226-483c-bef1-172c1bb9d5b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:02.451177 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10596e78-a226-483c-bef1-172c1bb9d5b2 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:02.464629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1125/2272] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:08:02.674755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b078131-7ce6-4445-b127-95270ddc8fff None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:02.675050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b078131-7ce6-4445-b127-95270ddc8fff None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:02.675565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b078131-7ce6-4445-b127-95270ddc8fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:02.685787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:02.715666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3b078131-7ce6-4445-b127-95270ddc8fff None None] MFA Rules not processed for user `afbb1d9c9bb34fbd8bfc78710c86d1f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:02.717335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3b078131-7ce6-4445-b127-95270ddc8fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:02.834643 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1148/2273] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:08:02.836671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10596e78-a226-483c-bef1-172c1bb9d5b2 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:02.838149 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-10596e78-a226-483c-bef1-172c1bb9d5b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:02.860056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10596e78-a226-483c-bef1-172c1bb9d5b2 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:02.876968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd20c01-137d-4cc5-b77a-9942d7ca48a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:02.877606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd20c01-137d-4cc5-b77a-9942d7ca48a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:02.877909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd20c01-137d-4cc5-b77a-9942d7ca48a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:02.878161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd20c01-137d-4cc5-b77a-9942d7ca48a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:02.880449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd20c01-137d-4cc5-b77a-9942d7ca48a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:02.901936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd20c01-137d-4cc5-b77a-9942d7ca48a0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:03.023002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1126/2274] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:08:02 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:08:03.027092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b078131-7ce6-4445-b127-95270ddc8fff None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:03.044536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b078131-7ce6-4445-b127-95270ddc8fff None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:03.161399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd20c01-137d-4cc5-b77a-9942d7ca48a0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:03.165412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4cd20c01-137d-4cc5-b77a-9942d7ca48a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:03.209912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b818-a9d4-42ea-bd2e-1982ba6783f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:03.213117 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b818-a9d4-42ea-bd2e-1982ba6783f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:03.213416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b818-a9d4-42ea-bd2e-1982ba6783f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:03.213701 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b818-a9d4-42ea-bd2e-1982ba6783f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:03.214362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92b818-a9d4-42ea-bd2e-1982ba6783f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:03.223151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92b818-a9d4-42ea-bd2e-1982ba6783f0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:03.297966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd20c01-137d-4cc5-b77a-9942d7ca48a0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:03.307518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:03.308462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:03.308881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:03.309432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] PATH_INFO: `/v3/projects/06a88fa4fff8413b997fd3b0603e9d42` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:03.311090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a88fa4fff8413b997fd3b0603e9d42)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:03.313326 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:03.377534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1149/2275] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:08:03 2022] POST /identity/v3/users => generated 296 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:08:03.378864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b818-a9d4-42ea-bd2e-1982ba6783f0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:03.392882 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b818-a9d4-42ea-bd2e-1982ba6783f0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:03.416940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8e870-c0ff-486a-a766-1a441313399b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:03.417625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8e870-c0ff-486a-a766-1a441313399b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:03.417899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8e870-c0ff-486a-a766-1a441313399b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:03.418172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8e870-c0ff-486a-a766-1a441313399b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:03.418867 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8e870-c0ff-486a-a766-1a441313399b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:03.420475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8e870-c0ff-486a-a766-1a441313399b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:03.445200 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1150/2276] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:08:03.449434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8e870-c0ff-486a-a766-1a441313399b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:03.463851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8e870-c0ff-486a-a766-1a441313399b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:03.486307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a88fa4fff8413b997fd3b0603e9d42', 'request_id': 'req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:03.495458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a88fa4fff8413b997fd3b0603e9d42', 'request_id': 'req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:03.516787 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a88fa4fff8413b997fd3b0603e9d42', 'request_id': 'req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:03.558247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2282584-6d69-4157-9204-cc9908b871b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:03.559174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2282584-6d69-4157-9204-cc9908b871b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:03.565104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2282584-6d69-4157-9204-cc9908b871b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:03.565104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2282584-6d69-4157-9204-cc9908b871b5 admin admin] PATH_INFO: `/v3/projects/8750ae6fd91c4232836b5360727849d5/users/6b1a524bce5847b3b08d0b841ced42f7/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:03.593362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2282584-6d69-4157-9204-cc9908b871b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8750ae6fd91c4232836b5360727849d5, user_id=6b1a524bce5847b3b08d0b841ced42f7, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:03.596274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2282584-6d69-4157-9204-cc9908b871b5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:03.604297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:03.654515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] The token cache is being invalidate because project 06a88fa4fff8413b997fd3b0603e9d42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:08:03.655932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:03.655932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:03.663939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1127/2277] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:08:03 2022] DELETE /identity/v3/projects/06a88fa4fff8413b997fd3b0603e9d42 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:08:03.664963 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:03.667931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:03.685437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac01a8-032d-48e9-a62c-f189d26d9bf1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:03.696679 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1151/2278] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:08:03 2022] PUT /identity/v3/projects/8750ae6fd91c4232836b5360727849d5/users/6b1a524bce5847b3b08d0b841ced42f7/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:08:03.702191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2282584-6d69-4157-9204-cc9908b871b5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:03.704289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a2282584-6d69-4157-9204-cc9908b871b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:03.795876 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:03.840278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9dbd4-1e26-42bd-9118-da7e26355f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:03.841009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9dbd4-1e26-42bd-9118-da7e26355f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:03.841487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9dbd4-1e26-42bd-9118-da7e26355f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:03.841953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9dbd4-1e26-42bd-9118-da7e26355f08 admin admin] PATH_INFO: `/v3/users/6f99dc80df784901892c91c8e89fdda4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:03.843563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9dbd4-1e26-42bd-9118-da7e26355f08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f99dc80df784901892c91c8e89fdda4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:03.845760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9dbd4-1e26-42bd-9118-da7e26355f08 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:03.852656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2282584-6d69-4157-9204-cc9908b871b5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:03.868253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a6358-e21a-415f-8ac0-87916c4940f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:03.869781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a6358-e21a-415f-8ac0-87916c4940f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:03.870646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a6358-e21a-415f-8ac0-87916c4940f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:03.870951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a6358-e21a-415f-8ac0-87916c4940f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:03.878864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a6358-e21a-415f-8ac0-87916c4940f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:03.878864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a6358-e21a-415f-8ac0-87916c4940f3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:03.889643 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1152/2279] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:08:03.891351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a6358-e21a-415f-8ac0-87916c4940f3 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:03.907789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a6358-e21a-415f-8ac0-87916c4940f3 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:03.908406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47176806-bee6-428e-b589-b7725cf7bf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:03.909066 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47176806-bee6-428e-b589-b7725cf7bf12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:03.909346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47176806-bee6-428e-b589-b7725cf7bf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:03.909694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47176806-bee6-428e-b589-b7725cf7bf12 admin admin] PATH_INFO: `/v3/projects/8750ae6fd91c4232836b5360727849d5/users/6b1a524bce5847b3b08d0b841ced42f7/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:03.912396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47176806-bee6-428e-b589-b7725cf7bf12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8750ae6fd91c4232836b5360727849d5, user_id=6b1a524bce5847b3b08d0b841ced42f7, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:03.915365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47176806-bee6-428e-b589-b7725cf7bf12 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:03.926657 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1153/2280] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:08:03 2022] PUT /identity/v3/projects/8750ae6fd91c4232836b5360727849d5/users/6b1a524bce5847b3b08d0b841ced42f7/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:08:04.094417 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-75e9dbd4-1e26-42bd-9118-da7e26355f08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f99dc80df784901892c91c8e89fdda4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:04.115879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-75e9dbd4-1e26-42bd-9118-da7e26355f08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f99dc80df784901892c91c8e89fdda4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:04.132348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-75e9dbd4-1e26-42bd-9118-da7e26355f08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f99dc80df784901892c91c8e89fdda4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:04.201710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-75e9dbd4-1e26-42bd-9118-da7e26355f08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f99dc80df784901892c91c8e89fdda4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:04.202340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-75e9dbd4-1e26-42bd-9118-da7e26355f08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f99dc80df784901892c91c8e89fdda4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:04.209578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1128/2281] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:08:03 2022] DELETE /identity/v3/users/6f99dc80df784901892c91c8e89fdda4 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:08:05.367716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47176806-bee6-428e-b589-b7725cf7bf12 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:05.370360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-47176806-bee6-428e-b589-b7725cf7bf12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:05.519053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47176806-bee6-428e-b589-b7725cf7bf12 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:05.621478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:05.622207 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:05.622578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:05.628442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] PATH_INFO: `/v3/projects/0c295a9da79b4eb695dd84f6ae3559b0` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:05.628442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c295a9da79b4eb695dd84f6ae3559b0)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:05.628442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:05.767831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=2,keystone:SELECT=97,keystone:DELETE=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:05.813259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c295a9da79b4eb695dd84f6ae3559b0', 'request_id': 'req-a221f124-ef8b-4887-9348-5d333470cdf5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:05.837861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c295a9da79b4eb695dd84f6ae3559b0', 'request_id': 'req-a221f124-ef8b-4887-9348-5d333470cdf5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:05.848997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c295a9da79b4eb695dd84f6ae3559b0', 'request_id': 'req-a221f124-ef8b-4887-9348-5d333470cdf5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:05.966777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] The token cache is being invalidate because project 0c295a9da79b4eb695dd84f6ae3559b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:08:05.967078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:05.975193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a221f124-ef8b-4887-9348-5d333470cdf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:05.977127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1154/2282] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:08:05 2022] DELETE /identity/v3/projects/0c295a9da79b4eb695dd84f6ae3559b0 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:08:05.982501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fab97-1388-47e1-8c05-ce41b8008465 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:05.983008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fab97-1388-47e1-8c05-ce41b8008465 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:05.983094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fab97-1388-47e1-8c05-ce41b8008465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:06.017593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-006fab97-1388-47e1-8c05-ce41b8008465 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:06.020998 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-006fab97-1388-47e1-8c05-ce41b8008465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:06.155696 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1129/2283] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:08:06.157280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019fdf0-f9a5-4969-9939-1405e3d8cb57 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:06.157965 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019fdf0-f9a5-4969-9939-1405e3d8cb57 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:06.158402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019fdf0-f9a5-4969-9939-1405e3d8cb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:06.217062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4019fdf0-f9a5-4969-9939-1405e3d8cb57 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:06.218720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4019fdf0-f9a5-4969-9939-1405e3d8cb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:06.242527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1155/2284] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:08:06.249070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fab97-1388-47e1-8c05-ce41b8008465 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:06.271731 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fab97-1388-47e1-8c05-ce41b8008465 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:06.277203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed34dd8-1bb2-405f-a857-dca4cbc06f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:06.278030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed34dd8-1bb2-405f-a857-dca4cbc06f0a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:06.278220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed34dd8-1bb2-405f-a857-dca4cbc06f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:06.278494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed34dd8-1bb2-405f-a857-dca4cbc06f0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:06.279597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed34dd8-1bb2-405f-a857-dca4cbc06f0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:08:06.279894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed34dd8-1bb2-405f-a857-dca4cbc06f0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:06.281853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed34dd8-1bb2-405f-a857-dca4cbc06f0a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:06.318580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1130/2285] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:08:06.320193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019fdf0-f9a5-4969-9939-1405e3d8cb57 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:06.333657 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019fdf0-f9a5-4969-9939-1405e3d8cb57 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:06.343281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b97c8-6693-4a18-8130-69c7003bc94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:06.344133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b97c8-6693-4a18-8130-69c7003bc94a admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:06.344491 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b97c8-6693-4a18-8130-69c7003bc94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:06.344864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b97c8-6693-4a18-8130-69c7003bc94a admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:06.345726 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b97c8-6693-4a18-8130-69c7003bc94a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:06.350324 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b97c8-6693-4a18-8130-69c7003bc94a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:06.474902 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1156/2286] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:08:06 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:08:06.480629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed34dd8-1bb2-405f-a857-dca4cbc06f0a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:06.535895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed34dd8-1bb2-405f-a857-dca4cbc06f0a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:06.653022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:06.680144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c3954-0803-4234-b5a7-f09081596d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:06.680776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c3954-0803-4234-b5a7-f09081596d63 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:06.681058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c3954-0803-4234-b5a7-f09081596d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:06.681335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c3954-0803-4234-b5a7-f09081596d63 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:06.681985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c3954-0803-4234-b5a7-f09081596d63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:06.683958 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c3954-0803-4234-b5a7-f09081596d63 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:06.789093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1131/2287] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:08:06 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:08:06.793542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b97c8-6693-4a18-8130-69c7003bc94a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:06.812396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b97c8-6693-4a18-8130-69c7003bc94a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:06.820753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8f7ba-4569-45a2-b556-1e3fbc4a0913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:06.821390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f7ba-4569-45a2-b556-1e3fbc4a0913 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:06.821675 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f7ba-4569-45a2-b556-1e3fbc4a0913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:06.821952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f7ba-4569-45a2-b556-1e3fbc4a0913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:06.822616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8f7ba-4569-45a2-b556-1e3fbc4a0913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:06.824267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8f7ba-4569-45a2-b556-1e3fbc4a0913 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:06.854608 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1157/2288] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:08:06.858635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c3954-0803-4234-b5a7-f09081596d63 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:06.873892 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c3954-0803-4234-b5a7-f09081596d63 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:06.887436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11651b65-1d25-4aeb-9924-a7f95dceccc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:06.888083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651b65-1d25-4aeb-9924-a7f95dceccc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:06.888367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651b65-1d25-4aeb-9924-a7f95dceccc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:06.888649 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651b65-1d25-4aeb-9924-a7f95dceccc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:06.889318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11651b65-1d25-4aeb-9924-a7f95dceccc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:06.892221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11651b65-1d25-4aeb-9924-a7f95dceccc5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:06.913542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1132/2289] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:08:06.918104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8f7ba-4569-45a2-b556-1e3fbc4a0913 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:06.937000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8f7ba-4569-45a2-b556-1e3fbc4a0913 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:06.954546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b52cb5-e55d-4a14-ac00-ee2653048e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:06.955305 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b52cb5-e55d-4a14-ac00-ee2653048e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:06.955549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b52cb5-e55d-4a14-ac00-ee2653048e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:06.956433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b52cb5-e55d-4a14-ac00-ee2653048e72 admin admin] PATH_INFO: `/v3/projects/c6aad87667ae4c4ab54f4b1de5e5536b/users/5cb9856f47b542b2890d6c0d5be0c9f4/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:06.980992 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b52cb5-e55d-4a14-ac00-ee2653048e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6aad87667ae4c4ab54f4b1de5e5536b, user_id=5cb9856f47b542b2890d6c0d5be0c9f4, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:06.985024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b52cb5-e55d-4a14-ac00-ee2653048e72 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:07.076876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1158/2290] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:08:06 2022] PUT /identity/v3/projects/c6aad87667ae4c4ab54f4b1de5e5536b/users/5cb9856f47b542b2890d6c0d5be0c9f4/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:08:07.091988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f44bb7-02c5-45ab-9209-dddf1ad9cb77 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:07.092535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f44bb7-02c5-45ab-9209-dddf1ad9cb77 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:07.092535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f44bb7-02c5-45ab-9209-dddf1ad9cb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:07.170624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e9f44bb7-02c5-45ab-9209-dddf1ad9cb77 None None] MFA Rules not processed for user `5cb9856f47b542b2890d6c0d5be0c9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:07.176535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e9f44bb7-02c5-45ab-9209-dddf1ad9cb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:07.290279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1133/2291] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:08:07.297361 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b52cb5-e55d-4a14-ac00-ee2653048e72 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:07.455478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b52cb5-e55d-4a14-ac00-ee2653048e72 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:07.464522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f7623-9fa5-4163-9ac2-8dd0366b2845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:07.465219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f7623-9fa5-4163-9ac2-8dd0366b2845 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:07.465508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f7623-9fa5-4163-9ac2-8dd0366b2845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:07.465771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f7623-9fa5-4163-9ac2-8dd0366b2845 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:07.466954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f7623-9fa5-4163-9ac2-8dd0366b2845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:07.471679 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f7623-9fa5-4163-9ac2-8dd0366b2845 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:07.601156 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1159/2292] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:08:07 2022] POST /identity/v3/projects => generated 381 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:08:07.602274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f44bb7-02c5-45ab-9209-dddf1ad9cb77 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:07.619389 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f44bb7-02c5-45ab-9209-dddf1ad9cb77 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:07.744878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea19193-8231-4010-8d1e-a6562c79c95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:07.745736 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea19193-8231-4010-8d1e-a6562c79c95c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:07.746187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea19193-8231-4010-8d1e-a6562c79c95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:07.746617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea19193-8231-4010-8d1e-a6562c79c95c admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:07.747467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea19193-8231-4010-8d1e-a6562c79c95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:07.757862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea19193-8231-4010-8d1e-a6562c79c95c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:07.876673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1134/2293] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:08:07 2022] POST /identity/v3/users => generated 297 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:08:07.881572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f7623-9fa5-4163-9ac2-8dd0366b2845 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:07.892086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f7623-9fa5-4163-9ac2-8dd0366b2845 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:07.902762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1182d0-d84e-4926-8829-3458ad548db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:07.903874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1182d0-d84e-4926-8829-3458ad548db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:07.903874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1182d0-d84e-4926-8829-3458ad548db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:07.907078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1182d0-d84e-4926-8829-3458ad548db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:07.907078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1182d0-d84e-4926-8829-3458ad548db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:07.907078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1182d0-d84e-4926-8829-3458ad548db3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:07.933151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1160/2294] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:08:07.934836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea19193-8231-4010-8d1e-a6562c79c95c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:07.949954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea19193-8231-4010-8d1e-a6562c79c95c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:07.956664 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1a615-dd24-4706-9a10-1cd202e8b528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:07.958497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1a615-dd24-4706-9a10-1cd202e8b528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:07.959033 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1a615-dd24-4706-9a10-1cd202e8b528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:07.959679 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1a615-dd24-4706-9a10-1cd202e8b528 admin admin] PATH_INFO: `/v3/projects/a179e4c567644bb4ab23a2c3954cd687/users/d8fa2efa3a7e48ea8f863d362f726529/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:08.000268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1a615-dd24-4706-9a10-1cd202e8b528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a179e4c567644bb4ab23a2c3954cd687, user_id=d8fa2efa3a7e48ea8f863d362f726529, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:08.005353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1a615-dd24-4706-9a10-1cd202e8b528 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:08.104083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1135/2295] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:08:07 2022] PUT /identity/v3/projects/a179e4c567644bb4ab23a2c3954cd687/users/d8fa2efa3a7e48ea8f863d362f726529/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:08:08.106100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1182d0-d84e-4926-8829-3458ad548db3 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:08.145269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1182d0-d84e-4926-8829-3458ad548db3 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:08.204599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:08.293985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451356a-0e95-46b3-8410-ba11c3c46dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:08.294630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451356a-0e95-46b3-8410-ba11c3c46dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:08.295210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451356a-0e95-46b3-8410-ba11c3c46dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:08.295493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451356a-0e95-46b3-8410-ba11c3c46dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:08.296525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451356a-0e95-46b3-8410-ba11c3c46dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:08.297814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451356a-0e95-46b3-8410-ba11c3c46dfc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:08.314118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1161/2296] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:08:08.315432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1a615-dd24-4706-9a10-1cd202e8b528 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:08.335118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1a615-dd24-4706-9a10-1cd202e8b528 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:08.346783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f399075-8faf-475b-a0c9-4a93dddea830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:08.348083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f399075-8faf-475b-a0c9-4a93dddea830 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:08.348555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f399075-8faf-475b-a0c9-4a93dddea830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:08.349744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f399075-8faf-475b-a0c9-4a93dddea830 admin admin] PATH_INFO: `/v3/projects/a179e4c567644bb4ab23a2c3954cd687/users/d8fa2efa3a7e48ea8f863d362f726529/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:08.356543 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f399075-8faf-475b-a0c9-4a93dddea830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a179e4c567644bb4ab23a2c3954cd687, user_id=d8fa2efa3a7e48ea8f863d362f726529, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:08.359991 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f399075-8faf-475b-a0c9-4a93dddea830 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:08.398636 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1136/2297] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:08:08 2022] PUT /identity/v3/projects/a179e4c567644bb4ab23a2c3954cd687/users/d8fa2efa3a7e48ea8f863d362f726529/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:08:08.400503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349eae11-a97c-4f2d-8ad4-8478c4d2bc7b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:08.400800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349eae11-a97c-4f2d-8ad4-8478c4d2bc7b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:08.401086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349eae11-a97c-4f2d-8ad4-8478c4d2bc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:08.448022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-349eae11-a97c-4f2d-8ad4-8478c4d2bc7b None None] MFA Rules not processed for user `d8fa2efa3a7e48ea8f863d362f726529`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:08.448022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-349eae11-a97c-4f2d-8ad4-8478c4d2bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:08.569317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1162/2298] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:08:08.585581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f399075-8faf-475b-a0c9-4a93dddea830 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:08.587319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5f399075-8faf-475b-a0c9-4a93dddea830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:08.620497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:08.676832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f399075-8faf-475b-a0c9-4a93dddea830 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:08.681231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecd6cd-8744-4e7f-90c5-3ddee1460603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:08.682394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecd6cd-8744-4e7f-90c5-3ddee1460603 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:08.682539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecd6cd-8744-4e7f-90c5-3ddee1460603 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:08.682865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecd6cd-8744-4e7f-90c5-3ddee1460603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:08.802546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecd6cd-8744-4e7f-90c5-3ddee1460603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:08.804083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecd6cd-8744-4e7f-90c5-3ddee1460603 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:08.813568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1137/2299] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:08:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:08:12.246457 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489e126-d99b-414b-bd81-08368a561175 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:12.247044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489e126-d99b-414b-bd81-08368a561175 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:12.247044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489e126-d99b-414b-bd81-08368a561175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:12.277424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e489e126-d99b-414b-bd81-08368a561175 None None] MFA Rules not processed for user `6b1a524bce5847b3b08d0b841ced42f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:12.278962 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e489e126-d99b-414b-bd81-08368a561175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:12.353497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1163/2300] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:08:12.367884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecd6cd-8744-4e7f-90c5-3ddee1460603 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:12.369303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-43ecd6cd-8744-4e7f-90c5-3ddee1460603 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:12.447098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecd6cd-8744-4e7f-90c5-3ddee1460603 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:12.452996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9acb02-f627-4a3b-b9d1-8aadcedbbd87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:12.453603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9acb02-f627-4a3b-b9d1-8aadcedbbd87 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:12.453903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9acb02-f627-4a3b-b9d1-8aadcedbbd87 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:12.454174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9acb02-f627-4a3b-b9d1-8aadcedbbd87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:12.455890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-df9acb02-f627-4a3b-b9d1-8aadcedbbd87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:12.517009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9acb02-f627-4a3b-b9d1-8aadcedbbd87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:12.518809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9acb02-f627-4a3b-b9d1-8aadcedbbd87 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:12.529012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1138/2301] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:08:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:08:13.965292 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489e126-d99b-414b-bd81-08368a561175 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:13.967249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e489e126-d99b-414b-bd81-08368a561175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:14.005773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:14.030486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489e126-d99b-414b-bd81-08368a561175 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:14.034656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aeb8a-0644-4a2d-927a-f2cb1dc2911c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:14.036011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aeb8a-0644-4a2d-927a-f2cb1dc2911c service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:14.036297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aeb8a-0644-4a2d-927a-f2cb1dc2911c service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:14.036587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46aeb8a-0644-4a2d-927a-f2cb1dc2911c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:14.038368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a46aeb8a-0644-4a2d-927a-f2cb1dc2911c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:14.110333 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46aeb8a-0644-4a2d-927a-f2cb1dc2911c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:14.111726 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46aeb8a-0644-4a2d-927a-f2cb1dc2911c service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:14.125250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1164/2302] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:08:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:08:14.532868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9acb02-f627-4a3b-b9d1-8aadcedbbd87 service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:14.532868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-df9acb02-f627-4a3b-b9d1-8aadcedbbd87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:14.546464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9acb02-f627-4a3b-b9d1-8aadcedbbd87 service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:14.549938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10058d-7a3c-48b3-81b1-2f631d03d305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:14.550579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10058d-7a3c-48b3-81b1-2f631d03d305 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:14.551939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10058d-7a3c-48b3-81b1-2f631d03d305 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:14.551939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10058d-7a3c-48b3-81b1-2f631d03d305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:14.559091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:14.566938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10058d-7a3c-48b3-81b1-2f631d03d305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:14.566938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10058d-7a3c-48b3-81b1-2f631d03d305 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:14.576755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1139/2303] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:08:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:08:20.824378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aeb8a-0644-4a2d-927a-f2cb1dc2911c service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:20.895635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46aeb8a-0644-4a2d-927a-f2cb1dc2911c service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:20.900223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:20.900976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:20.901255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:20.901530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] PATH_INFO: `/v3/users/5cb9856f47b542b2890d6c0d5be0c9f4` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:20.903245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cb9856f47b542b2890d6c0d5be0c9f4)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:20.905174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:21.134318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cb9856f47b542b2890d6c0d5be0c9f4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:21.147006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cb9856f47b542b2890d6c0d5be0c9f4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:21.155123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cb9856f47b542b2890d6c0d5be0c9f4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:21.210555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cb9856f47b542b2890d6c0d5be0c9f4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:21.211688 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cb9856f47b542b2890d6c0d5be0c9f4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:21.214321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:21.219068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1165/2304] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:08:20 2022] DELETE /identity/v3/users/5cb9856f47b542b2890d6c0d5be0c9f4 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:08:21.224227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9952c-4920-4b89-8b8b-2b132c322046 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:21.232343 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9952c-4920-4b89-8b8b-2b132c322046 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:21.232343 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9952c-4920-4b89-8b8b-2b132c322046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:21.244250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0ce9952c-4920-4b89-8b8b-2b132c322046 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:21.245462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0ce9952c-4920-4b89-8b8b-2b132c322046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:21.311397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1140/2305] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:08:21.320942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:21.322815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:21.386113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7f2d9-dfd8-420e-b0ec-322fbaff7265 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:21.389899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d2d47-c4c2-4ee3-a1f4-7c7227e2a5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:21.390589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d2d47-c4c2-4ee3-a1f4-7c7227e2a5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:21.390969 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d2d47-c4c2-4ee3-a1f4-7c7227e2a5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:21.391313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d2d47-c4c2-4ee3-a1f4-7c7227e2a5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:21.404485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d2d47-c4c2-4ee3-a1f4-7c7227e2a5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:21.406582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d2d47-c4c2-4ee3-a1f4-7c7227e2a5d5 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:21.419465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1166/2306] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:08:21.891099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9952c-4920-4b89-8b8b-2b132c322046 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:21.892575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0ce9952c-4920-4b89-8b8b-2b132c322046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:21.907522 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9952c-4920-4b89-8b8b-2b132c322046 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:21.912213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:21.915638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:21.915638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:21.915638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] PATH_INFO: `/v3/projects/c6aad87667ae4c4ab54f4b1de5e5536b` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:21.915638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6aad87667ae4c4ab54f4b1de5e5536b)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:21.923188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:22.035754 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6aad87667ae4c4ab54f4b1de5e5536b', 'request_id': 'req-ba15de15-1be1-4764-98d2-5cac56a818df'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:22.051289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6aad87667ae4c4ab54f4b1de5e5536b', 'request_id': 'req-ba15de15-1be1-4764-98d2-5cac56a818df'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:22.067955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6aad87667ae4c4ab54f4b1de5e5536b', 'request_id': 'req-ba15de15-1be1-4764-98d2-5cac56a818df'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:22.138554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] The token cache is being invalidate because project c6aad87667ae4c4ab54f4b1de5e5536b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:08:22.138889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:22.139656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:22.142142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1141/2307] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:08:21 2022] DELETE /identity/v3/projects/c6aad87667ae4c4ab54f4b1de5e5536b => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:08:22.146403 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d2d47-c4c2-4ee3-a1f4-7c7227e2a5d5 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:22.157938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d2d47-c4c2-4ee3-a1f4-7c7227e2a5d5 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:22.258975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe92bf-220f-4fb7-8950-05dfc71c192a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:22.259768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe92bf-220f-4fb7-8950-05dfc71c192a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:22.260125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe92bf-220f-4fb7-8950-05dfc71c192a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:22.260559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe92bf-220f-4fb7-8950-05dfc71c192a admin admin] PATH_INFO: `/v3/users/d8fa2efa3a7e48ea8f863d362f726529` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:22.261916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe92bf-220f-4fb7-8950-05dfc71c192a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8fa2efa3a7e48ea8f863d362f726529)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:22.265227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe92bf-220f-4fb7-8950-05dfc71c192a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:22.495784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e8fe92bf-220f-4fb7-8950-05dfc71c192a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8fa2efa3a7e48ea8f863d362f726529'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:22.509887 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e8fe92bf-220f-4fb7-8950-05dfc71c192a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8fa2efa3a7e48ea8f863d362f726529'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:22.517281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e8fe92bf-220f-4fb7-8950-05dfc71c192a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8fa2efa3a7e48ea8f863d362f726529'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:22.589935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e8fe92bf-220f-4fb7-8950-05dfc71c192a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8fa2efa3a7e48ea8f863d362f726529'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:22.590555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e8fe92bf-220f-4fb7-8950-05dfc71c192a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8fa2efa3a7e48ea8f863d362f726529'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:22.597059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1167/2308] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:08:22 2022] DELETE /identity/v3/users/d8fa2efa3a7e48ea8f863d362f726529 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:08:23.369200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:23.370819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:23.385439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15de15-1be1-4764-98d2-5cac56a818df admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:23.453902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:23.454922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:23.455515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:23.456058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] PATH_INFO: `/v3/projects/a179e4c567644bb4ab23a2c3954cd687` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:23.457595 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a179e4c567644bb4ab23a2c3954cd687)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:23.460062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:23.489910 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:23.597270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a179e4c567644bb4ab23a2c3954cd687', 'request_id': 'req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:23.604200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a179e4c567644bb4ab23a2c3954cd687', 'request_id': 'req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:23.610527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a179e4c567644bb4ab23a2c3954cd687', 'request_id': 'req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:23.678535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] The token cache is being invalidate because project a179e4c567644bb4ab23a2c3954cd687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:08:23.678973 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:23.680188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ceea7de7-1eda-4bfd-9d56-99ee1881f26e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:08:23.682773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1142/2309] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:08:23 2022] DELETE /identity/v3/projects/a179e4c567644bb4ab23a2c3954cd687 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:08:23.688259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1560c3-527f-41d1-96a0-66245cdb2b73 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:23.689063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1560c3-527f-41d1-96a0-66245cdb2b73 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:23.689509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1560c3-527f-41d1-96a0-66245cdb2b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:23.718492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6e1560c3-527f-41d1-96a0-66245cdb2b73 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:23.720093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6e1560c3-527f-41d1-96a0-66245cdb2b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:23.751573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:23.797437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1168/2310] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:08:23.806479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e6f6c-a011-4113-9642-4a5fbe258656 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:23.806802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e6f6c-a011-4113-9642-4a5fbe258656 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:23.807079 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e6f6c-a011-4113-9642-4a5fbe258656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:23.840796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ca0e6f6c-a011-4113-9642-4a5fbe258656 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:23.842736 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ca0e6f6c-a011-4113-9642-4a5fbe258656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:23.852265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1143/2311] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:08:23.856193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1560c3-527f-41d1-96a0-66245cdb2b73 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:23.872689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1560c3-527f-41d1-96a0-66245cdb2b73 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:23.881026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08e7d2-0efa-411b-aa5c-3235fe8e4513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:23.881829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08e7d2-0efa-411b-aa5c-3235fe8e4513 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:23.882631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08e7d2-0efa-411b-aa5c-3235fe8e4513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:23.882631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08e7d2-0efa-411b-aa5c-3235fe8e4513 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:23.883246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08e7d2-0efa-411b-aa5c-3235fe8e4513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:08:23.883557 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08e7d2-0efa-411b-aa5c-3235fe8e4513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:23.885223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08e7d2-0efa-411b-aa5c-3235fe8e4513 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:23.908994 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1169/2312] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:08:23.909862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e6f6c-a011-4113-9642-4a5fbe258656 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:23.920965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e6f6c-a011-4113-9642-4a5fbe258656 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:23.927535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606e2ed-ebb0-404a-8c89-6153a0044483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:23.928251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606e2ed-ebb0-404a-8c89-6153a0044483 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:23.928648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606e2ed-ebb0-404a-8c89-6153a0044483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:23.929579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606e2ed-ebb0-404a-8c89-6153a0044483 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:23.929711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606e2ed-ebb0-404a-8c89-6153a0044483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:23.931797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606e2ed-ebb0-404a-8c89-6153a0044483 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:24.034182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1144/2313] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:08:23 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:08:24.039046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08e7d2-0efa-411b-aa5c-3235fe8e4513 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:24.049815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08e7d2-0efa-411b-aa5c-3235fe8e4513 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:24.115440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4c2de-adb9-49be-8c91-b27a1cc80620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:24.116714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4c2de-adb9-49be-8c91-b27a1cc80620 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:24.117009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4c2de-adb9-49be-8c91-b27a1cc80620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:24.118046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4c2de-adb9-49be-8c91-b27a1cc80620 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:24.118242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4c2de-adb9-49be-8c91-b27a1cc80620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:24.120447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4c2de-adb9-49be-8c91-b27a1cc80620 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:24.225718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1170/2314] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:08:24 2022] POST /identity/v3/users => generated 283 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:08:24.229953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606e2ed-ebb0-404a-8c89-6153a0044483 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:24.241352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606e2ed-ebb0-404a-8c89-6153a0044483 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:24.247171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca869847-a419-445f-9fea-f7e6ce4eb3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:24.247895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca869847-a419-445f-9fea-f7e6ce4eb3df admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:24.248282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca869847-a419-445f-9fea-f7e6ce4eb3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:24.248621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca869847-a419-445f-9fea-f7e6ce4eb3df admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:24.249397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca869847-a419-445f-9fea-f7e6ce4eb3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:24.251153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca869847-a419-445f-9fea-f7e6ce4eb3df admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:24.265887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1145/2315] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:08:24.266858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4c2de-adb9-49be-8c91-b27a1cc80620 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:24.278613 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4c2de-adb9-49be-8c91-b27a1cc80620 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:24.286604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fe146-6d7c-4bba-9542-66b5646664ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:24.286604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe146-6d7c-4bba-9542-66b5646664ca admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:24.286604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe146-6d7c-4bba-9542-66b5646664ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:24.286604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe146-6d7c-4bba-9542-66b5646664ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:24.287038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fe146-6d7c-4bba-9542-66b5646664ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:24.287241 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fe146-6d7c-4bba-9542-66b5646664ca admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:24.303022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1171/2316] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:08:24.304587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca869847-a419-445f-9fea-f7e6ce4eb3df admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:24.316992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca869847-a419-445f-9fea-f7e6ce4eb3df admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:24.322289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c646c-703e-4aa5-a2d2-d7b5e76a1c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:24.323058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c646c-703e-4aa5-a2d2-d7b5e76a1c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:24.323371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c646c-703e-4aa5-a2d2-d7b5e76a1c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:24.323650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c646c-703e-4aa5-a2d2-d7b5e76a1c4b admin admin] PATH_INFO: `/v3/projects/c05ca075ba6546dbb260e009a167a549/users/01373ecd16c04ca696fd1375264c264f/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:24.341242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c646c-703e-4aa5-a2d2-d7b5e76a1c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c05ca075ba6546dbb260e009a167a549, user_id=01373ecd16c04ca696fd1375264c264f, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:24.344029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c646c-703e-4aa5-a2d2-d7b5e76a1c4b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:24.397335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1146/2317] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:08:24 2022] PUT /identity/v3/projects/c05ca075ba6546dbb260e009a167a549/users/01373ecd16c04ca696fd1375264c264f/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:08:24.473542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aae127-13b8-4c6a-8d92-f2246dd4e403 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:24.474049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aae127-13b8-4c6a-8d92-f2246dd4e403 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:24.474984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aae127-13b8-4c6a-8d92-f2246dd4e403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:24.508973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-11aae127-13b8-4c6a-8d92-f2246dd4e403 None None] MFA Rules not processed for user `01373ecd16c04ca696fd1375264c264f`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:24.510378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-11aae127-13b8-4c6a-8d92-f2246dd4e403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:24.593565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1172/2318] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:08:24.604941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c646c-703e-4aa5-a2d2-d7b5e76a1c4b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:24.606656 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-045c646c-703e-4aa5-a2d2-d7b5e76a1c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:24.667666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c646c-703e-4aa5-a2d2-d7b5e76a1c4b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:24.671314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac597bc-1938-422b-b917-72bc273dfab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:24.671923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac597bc-1938-422b-b917-72bc273dfab1 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:24.672206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac597bc-1938-422b-b917-72bc273dfab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:24.672480 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac597bc-1938-422b-b917-72bc273dfab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:24.681439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac597bc-1938-422b-b917-72bc273dfab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:24.683017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac597bc-1938-422b-b917-72bc273dfab1 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:24.693420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1147/2319] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:08:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:08:34.589520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:34.688899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:39.415969 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0745a4-9679-4f3e-8d4b-fe38f766db16 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:39.416599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0745a4-9679-4f3e-8d4b-fe38f766db16 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:39.416599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0745a4-9679-4f3e-8d4b-fe38f766db16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:39.436036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1b0745a4-9679-4f3e-8d4b-fe38f766db16 None None] MFA Rules not processed for user `afbb1d9c9bb34fbd8bfc78710c86d1f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:08:39.437335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1b0745a4-9679-4f3e-8d4b-fe38f766db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:39.492753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1173/2320] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:08:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:08:41.953601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac597bc-1938-422b-b917-72bc273dfab1 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:08:41.955078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2ac597bc-1938-422b-b917-72bc273dfab1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:42.037626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac597bc-1938-422b-b917-72bc273dfab1 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:08:42.042983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd11477-ca8b-49a2-b7e9-f1ecceea7173 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:08:42.043540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd11477-ca8b-49a2-b7e9-f1ecceea7173 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:08:42.043819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd11477-ca8b-49a2-b7e9-f1ecceea7173 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:08:42.044091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd11477-ca8b-49a2-b7e9-f1ecceea7173 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:08:42.046140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2cd11477-ca8b-49a2-b7e9-f1ecceea7173 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:08:42.059509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd11477-ca8b-49a2-b7e9-f1ecceea7173 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:08:42.061128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd11477-ca8b-49a2-b7e9-f1ecceea7173 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:08:42.072072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1148/2321] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:08:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:08:49.485740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=25 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:08:52.066936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=32 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:09:00.821175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0745a4-9679-4f3e-8d4b-fe38f766db16 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:09:00.822755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1b0745a4-9679-4f3e-8d4b-fe38f766db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:00.878832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0745a4-9679-4f3e-8d4b-fe38f766db16 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:09:00.882368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312455b0-0130-476c-8de0-0a7e541a54e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:09:00.883548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312455b0-0130-476c-8de0-0a7e541a54e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:00.883830 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312455b0-0130-476c-8de0-0a7e541a54e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:00.884107 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312455b0-0130-476c-8de0-0a7e541a54e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:00.885940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-312455b0-0130-476c-8de0-0a7e541a54e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:00.949245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312455b0-0130-476c-8de0-0a7e541a54e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:09:00.950736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312455b0-0130-476c-8de0-0a7e541a54e0 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:09:00.960044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1174/2322] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:09:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:09:06.504482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db88762-f479-4875-aebc-49802c6d5091 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:06.504972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db88762-f479-4875-aebc-49802c6d5091 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:06.505188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db88762-f479-4875-aebc-49802c6d5091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:06.525107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4db88762-f479-4875-aebc-49802c6d5091 None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:09:06.526297 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4db88762-f479-4875-aebc-49802c6d5091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:06.584204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1149/2323] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:09:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:09:06.629097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312455b0-0130-476c-8de0-0a7e541a54e0 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:09:06.630577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-312455b0-0130-476c-8de0-0a7e541a54e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:06.714235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312455b0-0130-476c-8de0-0a7e541a54e0 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:09:06.719160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c0da8-a201-4cf7-9802-b4c819d232af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:09:06.719767 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c0da8-a201-4cf7-9802-b4c819d232af service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:06.720085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c0da8-a201-4cf7-9802-b4c819d232af service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:06.720359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c0da8-a201-4cf7-9802-b4c819d232af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:06.729260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c0da8-a201-4cf7-9802-b4c819d232af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:09:06.730865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c0da8-a201-4cf7-9802-b4c819d232af service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:09:06.739106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1175/2324] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:09:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:09:07.960308 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db88762-f479-4875-aebc-49802c6d5091 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:09:08.036939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db88762-f479-4875-aebc-49802c6d5091 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:09:08.044633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:09:08.045100 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:08.045592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:08.045772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] PATH_INFO: `/v3/users/01373ecd16c04ca696fd1375264c264f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:08.046943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] RBAC: Authorizing `identity:delete_user(user_id=01373ecd16c04ca696fd1375264c264f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:09:08.049053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:09:08.226733 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01373ecd16c04ca696fd1375264c264f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:09:08.242214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01373ecd16c04ca696fd1375264c264f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:09:08.249668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01373ecd16c04ca696fd1375264c264f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:09:08.364680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01373ecd16c04ca696fd1375264c264f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:09:08.370483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01373ecd16c04ca696fd1375264c264f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:09:08.376963 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1150/2325] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:09:07 2022] DELETE /identity/v3/users/01373ecd16c04ca696fd1375264c264f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:09:08.381131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b6b76-48b6-4999-920b-4d291f9cef65 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:08.381660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b6b76-48b6-4999-920b-4d291f9cef65 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:08.381660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b6b76-48b6-4999-920b-4d291f9cef65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:08.422554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2d1b6b76-48b6-4999-920b-4d291f9cef65 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:09:08.423858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2d1b6b76-48b6-4999-920b-4d291f9cef65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:08.483476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:09:08.533538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1176/2326] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:09:08.544740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:09:08.546521 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:08.586401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:09:08.651843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d1d7b-6e0c-46b9-9765-2b4544d3836f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:09:08.664662 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749ccd0-90c5-46ba-b041-a52e84602a70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:09:08.666837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749ccd0-90c5-46ba-b041-a52e84602a70 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:08.666837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749ccd0-90c5-46ba-b041-a52e84602a70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:08.666837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749ccd0-90c5-46ba-b041-a52e84602a70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:08.676766 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749ccd0-90c5-46ba-b041-a52e84602a70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:09:08.679628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749ccd0-90c5-46ba-b041-a52e84602a70 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:09:08.692881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1151/2327] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:09:09.324224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b6b76-48b6-4999-920b-4d291f9cef65 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:09:09.325876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2d1b6b76-48b6-4999-920b-4d291f9cef65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:09.340807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b6b76-48b6-4999-920b-4d291f9cef65 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:09:09.346258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:09:09.347230 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:09.347755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:09.348294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] PATH_INFO: `/v3/projects/c05ca075ba6546dbb260e009a167a549` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:09.350020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c05ca075ba6546dbb260e009a167a549)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:09:09.352353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:09:09.519073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c05ca075ba6546dbb260e009a167a549', 'request_id': 'req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:09:09.526411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c05ca075ba6546dbb260e009a167a549', 'request_id': 'req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:09:09.534059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c05ca075ba6546dbb260e009a167a549', 'request_id': 'req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:09:09.651204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] The token cache is being invalidate because project c05ca075ba6546dbb260e009a167a549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:09:09.651515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:09:09.652324 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3c961dc0-f4cc-4b7a-9b47-dd915f4628f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:09:09.654738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1177/2328] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:09:09 2022] DELETE /identity/v3/projects/c05ca075ba6546dbb260e009a167a549 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:09:18.746008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=15 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:09:19.646801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:09:27.796336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587fac7-d6db-438d-9fac-315d43838623 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:27.797468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587fac7-d6db-438d-9fac-315d43838623 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:27.797605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587fac7-d6db-438d-9fac-315d43838623 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:27.798990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1152/2329] 10.222.0.59 () {54 vars in 921 bytes} [Fri Sep 16 08:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:09:27.805008 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb985c-e781-487b-ae62-91cf80450f71 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:27.805488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb985c-e781-487b-ae62-91cf80450f71 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:27.806268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb985c-e781-487b-ae62-91cf80450f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:27.829331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d5cb985c-e781-487b-ae62-91cf80450f71 None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:09:27.830760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d5cb985c-e781-487b-ae62-91cf80450f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:27.925382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1178/2330] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Sep 16 08:09:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:09:27.934223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587fac7-d6db-438d-9fac-315d43838623 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:09:27.936688 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b587fac7-d6db-438d-9fac-315d43838623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:28.008606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587fac7-d6db-438d-9fac-315d43838623 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:09:28.012305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9542f2-691c-492d-af8b-a3604c1d35eb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c214b2c97a7b41698b433eac8379442a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:09:28.012923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9542f2-691c-492d-af8b-a3604c1d35eb service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:28.013231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9542f2-691c-492d-af8b-a3604c1d35eb service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:28.013514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9542f2-691c-492d-af8b-a3604c1d35eb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:28.024389 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9542f2-691c-492d-af8b-a3604c1d35eb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:09:28.025870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9542f2-691c-492d-af8b-a3604c1d35eb service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:09:28.037621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1153/2331] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:09:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:09:28.214311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f69fb-54aa-4c38-9923-5b3aabef41d2 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:28.214604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f69fb-54aa-4c38-9923-5b3aabef41d2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:28.214903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f69fb-54aa-4c38-9923-5b3aabef41d2 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:28.216248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1179/2332] 10.222.0.59 () {54 vars in 921 bytes} [Fri Sep 16 08:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:09:28.223692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0db05-e481-4477-b25c-7912b9d434f3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:28.224091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0db05-e481-4477-b25c-7912b9d434f3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:28.224494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0db05-e481-4477-b25c-7912b9d434f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:28.247281 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-35f0db05-e481-4477-b25c-7912b9d434f3 None None] MFA Rules not processed for user `a29c4e53ac9a438e9701a70f3de3f32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:09:28.248603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-35f0db05-e481-4477-b25c-7912b9d434f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:28.304878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1154/2333] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Sep 16 08:09:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:09:28.316890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f69fb-54aa-4c38-9923-5b3aabef41d2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:09:28.318628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cd4f69fb-54aa-4c38-9923-5b3aabef41d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:28.331987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f69fb-54aa-4c38-9923-5b3aabef41d2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:09:28.335444 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d26076-c8db-4cc8-bb5d-eb9cbc61107e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:09:28.336031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26076-c8db-4cc8-bb5d-eb9cbc61107e service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:28.336307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26076-c8db-4cc8-bb5d-eb9cbc61107e service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:28.336610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26076-c8db-4cc8-bb5d-eb9cbc61107e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:28.344180 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d26076-c8db-4cc8-bb5d-eb9cbc61107e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:09:28.346493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d26076-c8db-4cc8-bb5d-eb9cbc61107e service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:09:28.355706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1180/2334] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:09:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:09:28.442646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62284e-a7c0-404b-b051-f5aeccbbbb10 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:28.443080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62284e-a7c0-404b-b051-f5aeccbbbb10 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:28.443425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62284e-a7c0-404b-b051-f5aeccbbbb10 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:28.444844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1155/2335] 10.222.0.59 () {54 vars in 921 bytes} [Fri Sep 16 08:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 08:09:28.450328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99173e33-d884-46a7-9f4c-c0f901b4300d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:28.450702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99173e33-d884-46a7-9f4c-c0f901b4300d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:28.451047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99173e33-d884-46a7-9f4c-c0f901b4300d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:28.471357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-99173e33-d884-46a7-9f4c-c0f901b4300d None None] MFA Rules not processed for user `e32b5bf1a2524f1cbc432f38195e4c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:09:28.472557 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-99173e33-d884-46a7-9f4c-c0f901b4300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:28.482282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1181/2336] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Sep 16 08:09:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:09:28.493344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62284e-a7c0-404b-b051-f5aeccbbbb10 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:09:28.495181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2c62284e-a7c0-404b-b051-f5aeccbbbb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:28.508730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62284e-a7c0-404b-b051-f5aeccbbbb10 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:09:28.512382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3084a0-390b-4ae5-b956-58a2f99ac7c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:09:28.512986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3084a0-390b-4ae5-b956-58a2f99ac7c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:28.513268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3084a0-390b-4ae5-b956-58a2f99ac7c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:28.513546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3084a0-390b-4ae5-b956-58a2f99ac7c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:28.524021 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3084a0-390b-4ae5-b956-58a2f99ac7c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:09:28.525936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3084a0-390b-4ae5-b956-58a2f99ac7c6 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:09:28.537019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1156/2337] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:09:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:09:32.853442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a0a9f-ad96-4573-9b98-b4897c965dcf None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:32.853931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a0a9f-ad96-4573-9b98-b4897c965dcf None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:32.854029 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a0a9f-ad96-4573-9b98-b4897c965dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:32.873327 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-069a0a9f-ad96-4573-9b98-b4897c965dcf None None] MFA Rules not processed for user `ea2f6b804e164dbcb4f67926f23fdd59`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:09:32.874515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-069a0a9f-ad96-4573-9b98-b4897c965dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:32.929531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1182/2338] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:09:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:09:38.048925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50544993-e7fc-4e42-bfc0-b873eb41559f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:38.049487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50544993-e7fc-4e42-bfc0-b873eb41559f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:38.049487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50544993-e7fc-4e42-bfc0-b873eb41559f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:38.069661 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-50544993-e7fc-4e42-bfc0-b873eb41559f None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:09:38.070917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-50544993-e7fc-4e42-bfc0-b873eb41559f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:38.128873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1157/2339] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:09:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:09:38.143190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a0a9f-ad96-4573-9b98-b4897c965dcf None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:09:38.144771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-069a0a9f-ad96-4573-9b98-b4897c965dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:38.219138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a0a9f-ad96-4573-9b98-b4897c965dcf None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:09:38.224630 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5792922-6080-46c7-bbfd-5fac784ccbc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:09:38.225250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5792922-6080-46c7-bbfd-5fac784ccbc4 service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:38.225554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5792922-6080-46c7-bbfd-5fac784ccbc4 service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:38.225828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5792922-6080-46c7-bbfd-5fac784ccbc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:38.235646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5792922-6080-46c7-bbfd-5fac784ccbc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:09:38.237288 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5792922-6080-46c7-bbfd-5fac784ccbc4 service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:09:38.245759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1183/2340] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:09:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 16 08:09:48.121880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=80 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:09:48.242662 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:09:58.939214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e00fb-cd94-4d1b-b767-16ed9f8ec338 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:09:58.939738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e00fb-cd94-4d1b-b767-16ed9f8ec338 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:09:58.939843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e00fb-cd94-4d1b-b767-16ed9f8ec338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:09:58.959806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b14e00fb-cd94-4d1b-b767-16ed9f8ec338 None None] MFA Rules not processed for user `5a85cff936184b5d94375fd103493950`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:09:58.961105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b14e00fb-cd94-4d1b-b767-16ed9f8ec338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:09:59.022400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1158/2341] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:09:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:10:03.829512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4c598-9058-4fce-8b85-7e76670670b1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:03.830004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4c598-9058-4fce-8b85-7e76670670b1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:03.830099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4c598-9058-4fce-8b85-7e76670670b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:03.849792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2de4c598-9058-4fce-8b85-7e76670670b1 None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:10:03.851048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2de4c598-9058-4fce-8b85-7e76670670b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:03.858565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1184/2342] 10.222.0.59 () {58 vars in 967 bytes} [Fri Sep 16 08:10:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:10:03.872537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e00fb-cd94-4d1b-b767-16ed9f8ec338 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:03.885304 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e00fb-cd94-4d1b-b767-16ed9f8ec338 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:03.890307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97742f3-bd19-4bb3-9058-dcb9a9b18695 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:03.890952 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97742f3-bd19-4bb3-9058-dcb9a9b18695 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:03.891234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97742f3-bd19-4bb3-9058-dcb9a9b18695 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:03.891514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97742f3-bd19-4bb3-9058-dcb9a9b18695 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:03.900907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97742f3-bd19-4bb3-9058-dcb9a9b18695 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:03.902519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97742f3-bd19-4bb3-9058-dcb9a9b18695 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:03.911698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1159/2343] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:10:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:10:13.375164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4c598-9058-4fce-8b85-7e76670670b1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:13.384492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4c598-9058-4fce-8b85-7e76670670b1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:13.388088 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be052c23-c425-4f7b-9b9d-add04d7244e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:13.388687 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be052c23-c425-4f7b-9b9d-add04d7244e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:13.388974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be052c23-c425-4f7b-9b9d-add04d7244e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:13.389247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be052c23-c425-4f7b-9b9d-add04d7244e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:13.391090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-be052c23-c425-4f7b-9b9d-add04d7244e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:13.404462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be052c23-c425-4f7b-9b9d-add04d7244e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:13.406179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be052c23-c425-4f7b-9b9d-add04d7244e4 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:13.417378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1185/2344] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:10:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:10:13.908089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=31 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:10:16.703612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97742f3-bd19-4bb3-9058-dcb9a9b18695 service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:16.712922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97742f3-bd19-4bb3-9058-dcb9a9b18695 service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:16.716736 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9f6ed-ff52-400b-ac2a-f3741712ff06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:16.717348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f6ed-ff52-400b-ac2a-f3741712ff06 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:16.717613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f6ed-ff52-400b-ac2a-f3741712ff06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:16.717908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f6ed-ff52-400b-ac2a-f3741712ff06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:16.719677 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2cc9f6ed-ff52-400b-ac2a-f3741712ff06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:16.733539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9f6ed-ff52-400b-ac2a-f3741712ff06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:16.735004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9f6ed-ff52-400b-ac2a-f3741712ff06 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:16.745851 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1160/2345] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:10:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:10:21.419929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a29d9-ec2b-4fdd-b905-8545c2af0f89 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:21.421417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a29d9-ec2b-4fdd-b905-8545c2af0f89 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:21.421417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a29d9-ec2b-4fdd-b905-8545c2af0f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:21.440192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3e9a29d9-ec2b-4fdd-b905-8545c2af0f89 None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:10:21.441405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3e9a29d9-ec2b-4fdd-b905-8545c2af0f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:21.449248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1186/2346] 10.222.0.59 () {58 vars in 967 bytes} [Fri Sep 16 08:10:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:10:21.471966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9f6ed-ff52-400b-ac2a-f3741712ff06 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:21.483977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9f6ed-ff52-400b-ac2a-f3741712ff06 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:21.493464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecf3d3-5726-4bf0-ab56-220b40a532b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:21.495986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf3d3-5726-4bf0-ab56-220b40a532b0 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:21.495986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf3d3-5726-4bf0-ab56-220b40a532b0 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:21.495986 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf3d3-5726-4bf0-ab56-220b40a532b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:21.504358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecf3d3-5726-4bf0-ab56-220b40a532b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:21.506131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecf3d3-5726-4bf0-ab56-220b40a532b0 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:21.515716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1161/2347] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:10:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:10:24.258070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418073d5-82b0-4b15-acb3-e7df7776f6c9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:24.258594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418073d5-82b0-4b15-acb3-e7df7776f6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:24.258792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418073d5-82b0-4b15-acb3-e7df7776f6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:24.278338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-418073d5-82b0-4b15-acb3-e7df7776f6c9 None None] MFA Rules not processed for user `393fddf9c75447fab334b84694dcf4f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:10:24.279907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-418073d5-82b0-4b15-acb3-e7df7776f6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:24.335726 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1187/2348] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:10:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:10:28.385373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecf3d3-5726-4bf0-ab56-220b40a532b0 service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:28.395727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecf3d3-5726-4bf0-ab56-220b40a532b0 service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:28.399782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383e3aa-a97c-46a4-99ae-4a0024e1102f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:28.404628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383e3aa-a97c-46a4-99ae-4a0024e1102f service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:28.404628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383e3aa-a97c-46a4-99ae-4a0024e1102f service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:28.404628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383e3aa-a97c-46a4-99ae-4a0024e1102f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:28.404628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6383e3aa-a97c-46a4-99ae-4a0024e1102f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:28.414082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383e3aa-a97c-46a4-99ae-4a0024e1102f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:28.415557 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383e3aa-a97c-46a4-99ae-4a0024e1102f service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:28.425100 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1162/2349] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:10:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:10:31.963094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418073d5-82b0-4b15-acb3-e7df7776f6c9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:31.968310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-418073d5-82b0-4b15-acb3-e7df7776f6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:31.970637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=38 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:10:32.035995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418073d5-82b0-4b15-acb3-e7df7776f6c9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:32.040493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:32.041272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:32.041558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:32.041883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] PATH_INFO: `/v3/users/66702831c1ca4158976920f6be2cc80c` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:32.043069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66702831c1ca4158976920f6be2cc80c)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:32.045107 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:32.225186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66702831c1ca4158976920f6be2cc80c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:32.237073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66702831c1ca4158976920f6be2cc80c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:32.244413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66702831c1ca4158976920f6be2cc80c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:32.308996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66702831c1ca4158976920f6be2cc80c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:32.309616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66702831c1ca4158976920f6be2cc80c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:32.315781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1188/2350] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:10:31 2022] DELETE /identity/v3/users/66702831c1ca4158976920f6be2cc80c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:10:32.555686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383e3aa-a97c-46a4-99ae-4a0024e1102f service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:32.568121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383e3aa-a97c-46a4-99ae-4a0024e1102f service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:32.625094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:32.625723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:32.626005 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:32.626282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] PATH_INFO: `/v3/projects/5870af1bbad64b138178c47e4a571d81` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:32.627469 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5870af1bbad64b138178c47e4a571d81)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:32.629366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:32.733012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5870af1bbad64b138178c47e4a571d81', 'request_id': 'req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:32.739584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5870af1bbad64b138178c47e4a571d81', 'request_id': 'req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:32.745453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5870af1bbad64b138178c47e4a571d81', 'request_id': 'req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:32.845840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] The token cache is being invalidate because project 5870af1bbad64b138178c47e4a571d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:10:32.846062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:32.846687 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:32.848191 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1163/2351] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:10:32 2022] DELETE /identity/v3/projects/5870af1bbad64b138178c47e4a571d81 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:10:32.852353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:32.853741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:32.867377 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de27f39-15a0-4989-b500-74dbbe737624 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:32.924625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:32.925258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:32.925540 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:32.925818 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] PATH_INFO: `/v3/users/e4107a0136834061828fb975136de88e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:32.927011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4107a0136834061828fb975136de88e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:32.929025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:32.950885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:10:33.140018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4107a0136834061828fb975136de88e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:33.153144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4107a0136834061828fb975136de88e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:33.160401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4107a0136834061828fb975136de88e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:33.234822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4107a0136834061828fb975136de88e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:33.235451 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4107a0136834061828fb975136de88e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:33.242049 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1189/2352] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:10:32 2022] DELETE /identity/v3/users/e4107a0136834061828fb975136de88e => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:10:33.487346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:33.488858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:33.502628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81849f3-ce9b-4c2c-a312-f6f3fd95baf2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:33.550078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:10:33.559901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:33.560818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:33.560818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:33.561103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] PATH_INFO: `/v3/projects/d5e9862850ba48d286fe4c4e52be7891` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:33.562205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e9862850ba48d286fe4c4e52be7891)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:33.564497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:33.688932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e9862850ba48d286fe4c4e52be7891', 'request_id': 'req-db67b6aa-e36f-42b8-850e-51c7d9481fe0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:33.697575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e9862850ba48d286fe4c4e52be7891', 'request_id': 'req-db67b6aa-e36f-42b8-850e-51c7d9481fe0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:33.705470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e9862850ba48d286fe4c4e52be7891', 'request_id': 'req-db67b6aa-e36f-42b8-850e-51c7d9481fe0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:33.787752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] The token cache is being invalidate because project d5e9862850ba48d286fe4c4e52be7891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:10:33.788102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:33.788667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:10:33.790368 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1164/2353] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:10:33 2022] DELETE /identity/v3/projects/d5e9862850ba48d286fe4c4e52be7891 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:10:41.450370 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:41.451982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:41.516013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347df225-c03f-4529-a16e-fbe70f5696dd admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:41.577141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88c396-5dac-4cdb-96bd-d11044c0ab3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:41.577680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88c396-5dac-4cdb-96bd-d11044c0ab3b service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:41.577962 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88c396-5dac-4cdb-96bd-d11044c0ab3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:41.578232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88c396-5dac-4cdb-96bd-d11044c0ab3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:41.580021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5f88c396-5dac-4cdb-96bd-d11044c0ab3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:41.589932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88c396-5dac-4cdb-96bd-d11044c0ab3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:41.591451 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88c396-5dac-4cdb-96bd-d11044c0ab3b service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:41.599486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1190/2354] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:10:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:10:43.784249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:10:44.144394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:44.145819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:44.221860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67b6aa-e36f-42b8-850e-51c7d9481fe0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:44.227679 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393ee8d-a7a0-4f00-b3a9-dd0d9614e49f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:44.231795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393ee8d-a7a0-4f00-b3a9-dd0d9614e49f service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:44.231795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393ee8d-a7a0-4f00-b3a9-dd0d9614e49f service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:44.231795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393ee8d-a7a0-4f00-b3a9-dd0d9614e49f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:44.231795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e393ee8d-a7a0-4f00-b3a9-dd0d9614e49f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:44.290022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393ee8d-a7a0-4f00-b3a9-dd0d9614e49f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:44.291663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393ee8d-a7a0-4f00-b3a9-dd0d9614e49f service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:44.300739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1165/2355] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:10:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:10:45.375160 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88c396-5dac-4cdb-96bd-d11044c0ab3b service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:45.376592 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5f88c396-5dac-4cdb-96bd-d11044c0ab3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:45.434980 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88c396-5dac-4cdb-96bd-d11044c0ab3b service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:45.438819 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3daf896-483d-40b3-96c5-246d47eed69e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:45.439558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daf896-483d-40b3-96c5-246d47eed69e service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:45.439848 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daf896-483d-40b3-96c5-246d47eed69e service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:45.440181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3daf896-483d-40b3-96c5-246d47eed69e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:45.442098 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d3daf896-483d-40b3-96c5-246d47eed69e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:45.460606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:10:45.508509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3daf896-483d-40b3-96c5-246d47eed69e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:45.509925 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3daf896-483d-40b3-96c5-246d47eed69e service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:45.519611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1191/2356] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:10:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:10:49.645999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393ee8d-a7a0-4f00-b3a9-dd0d9614e49f service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:49.658546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393ee8d-a7a0-4f00-b3a9-dd0d9614e49f service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:49.667749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4acc0-494b-411c-9ac4-c338ea591e29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:49.668669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4acc0-494b-411c-9ac4-c338ea591e29 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:49.669271 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4acc0-494b-411c-9ac4-c338ea591e29 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:49.669772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4acc0-494b-411c-9ac4-c338ea591e29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:49.671970 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e2f4acc0-494b-411c-9ac4-c338ea591e29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:49.688476 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4acc0-494b-411c-9ac4-c338ea591e29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:49.690102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4acc0-494b-411c-9ac4-c338ea591e29 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:49.705086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1166/2357] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:10:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:10:55.572204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=21 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:10:58.051470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e80fd4-df94-4125-8944-d8ac98145755 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:58.052007 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e80fd4-df94-4125-8944-d8ac98145755 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:58.052112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e80fd4-df94-4125-8944-d8ac98145755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:58.072150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-52e80fd4-df94-4125-8944-d8ac98145755 None None] MFA Rules not processed for user `81e89f8fb70d49d18cd0c81c98dd4add`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:10:58.073399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-52e80fd4-df94-4125-8944-d8ac98145755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:58.131116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1192/2358] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:10:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:10:58.440440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4acc0-494b-411c-9ac4-c338ea591e29 service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:58.441978 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e2f4acc0-494b-411c-9ac4-c338ea591e29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:58.455211 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4acc0-494b-411c-9ac4-c338ea591e29 service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:58.459020 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00079cc-9dba-40a9-874a-23b6fa90f6ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:58.459630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00079cc-9dba-40a9-874a-23b6fa90f6ee service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:58.460032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00079cc-9dba-40a9-874a-23b6fa90f6ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:58.460321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00079cc-9dba-40a9-874a-23b6fa90f6ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:58.462159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f00079cc-9dba-40a9-874a-23b6fa90f6ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:58.472683 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00079cc-9dba-40a9-874a-23b6fa90f6ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:58.474121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00079cc-9dba-40a9-874a-23b6fa90f6ee service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:58.483126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1167/2359] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:10:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:10:58.878357 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e80fd4-df94-4125-8944-d8ac98145755 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:10:58.879793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-52e80fd4-df94-4125-8944-d8ac98145755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:58.952995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e80fd4-df94-4125-8944-d8ac98145755 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:10:58.957893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44da714-3d56-4425-9cbc-186e74294159 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:10:58.958551 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44da714-3d56-4425-9cbc-186e74294159 service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:10:58.958972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44da714-3d56-4425-9cbc-186e74294159 service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:10:58.959253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44da714-3d56-4425-9cbc-186e74294159 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:10:58.961024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c44da714-3d56-4425-9cbc-186e74294159 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:10:58.974574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44da714-3d56-4425-9cbc-186e74294159 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:10:58.976245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44da714-3d56-4425-9cbc-186e74294159 service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:10:58.987092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1193/2360] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:10:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:04.268210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00079cc-9dba-40a9-874a-23b6fa90f6ee service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:04.270119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f00079cc-9dba-40a9-874a-23b6fa90f6ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:04.290019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00079cc-9dba-40a9-874a-23b6fa90f6ee service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:04.297341 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193d8d6-0b2c-4393-8569-a7d729ae89bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:04.298216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d8d6-0b2c-4393-8569-a7d729ae89bc service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:04.298624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d8d6-0b2c-4393-8569-a7d729ae89bc service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:04.299072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193d8d6-0b2c-4393-8569-a7d729ae89bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:04.301026 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7193d8d6-0b2c-4393-8569-a7d729ae89bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:04.320099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193d8d6-0b2c-4393-8569-a7d729ae89bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:04.321738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193d8d6-0b2c-4393-8569-a7d729ae89bc service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:04.334870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1168/2361] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:08.982352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=57 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:10.500382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614e9f8-a352-46d6-825e-805fef18efbd None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:10.500965 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614e9f8-a352-46d6-825e-805fef18efbd None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:10.502597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614e9f8-a352-46d6-825e-805fef18efbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:10.520461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a614e9f8-a352-46d6-825e-805fef18efbd None None] MFA Rules not processed for user `ea2f6b804e164dbcb4f67926f23fdd59`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:10.524100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a614e9f8-a352-46d6-825e-805fef18efbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:10.528542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1194/2362] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:10.914069 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193d8d6-0b2c-4393-8569-a7d729ae89bc service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:10.915521 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7193d8d6-0b2c-4393-8569-a7d729ae89bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:10.927974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193d8d6-0b2c-4393-8569-a7d729ae89bc service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:10.931627 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59406752-1f52-45f4-b06b-6521522fa471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:10.932221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59406752-1f52-45f4-b06b-6521522fa471 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:10.932500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59406752-1f52-45f4-b06b-6521522fa471 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:10.932773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59406752-1f52-45f4-b06b-6521522fa471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:10.934439 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-59406752-1f52-45f4-b06b-6521522fa471 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:10.944029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59406752-1f52-45f4-b06b-6521522fa471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:10.945520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59406752-1f52-45f4-b06b-6521522fa471 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:10.956361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1169/2363] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:13.909360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614e9f8-a352-46d6-825e-805fef18efbd None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:13.922243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614e9f8-a352-46d6-825e-805fef18efbd None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:13.926102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021e1b6-aa9e-4270-a8ea-2cae9b9bfd6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:13.926840 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021e1b6-aa9e-4270-a8ea-2cae9b9bfd6e service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:13.927213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021e1b6-aa9e-4270-a8ea-2cae9b9bfd6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:13.927573 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021e1b6-aa9e-4270-a8ea-2cae9b9bfd6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:13.929462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0021e1b6-aa9e-4270-a8ea-2cae9b9bfd6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:13.996113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021e1b6-aa9e-4270-a8ea-2cae9b9bfd6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:13.997604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021e1b6-aa9e-4270-a8ea-2cae9b9bfd6e service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:14.007622 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1195/2364] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:15.698897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59406752-1f52-45f4-b06b-6521522fa471 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:15.710540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=81 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:15.726247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59406752-1f52-45f4-b06b-6521522fa471 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:15.726247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d239366-513a-4535-b900-fb1e6d89a7d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:15.726247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d239366-513a-4535-b900-fb1e6d89a7d7 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:15.726247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d239366-513a-4535-b900-fb1e6d89a7d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:15.727287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d239366-513a-4535-b900-fb1e6d89a7d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:15.727855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9d239366-513a-4535-b900-fb1e6d89a7d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:15.745158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d239366-513a-4535-b900-fb1e6d89a7d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:15.748938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d239366-513a-4535-b900-fb1e6d89a7d7 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:15.761544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1170/2365] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:16.260714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb38f2-b809-48a0-ace4-739839f971e1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:16.261610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb38f2-b809-48a0-ace4-739839f971e1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:16.262061 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb38f2-b809-48a0-ace4-739839f971e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:16.286911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-52fb38f2-b809-48a0-ace4-739839f971e1 None None] MFA Rules not processed for user `afbb1d9c9bb34fbd8bfc78710c86d1f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:16.289911 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-52fb38f2-b809-48a0-ace4-739839f971e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:16.356454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1196/2366] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:19.129894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d239366-513a-4535-b900-fb1e6d89a7d7 service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:19.129894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9d239366-513a-4535-b900-fb1e6d89a7d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:19.142946 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d239366-513a-4535-b900-fb1e6d89a7d7 service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:19.147907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d725a83-1bf5-4e39-a724-d53eb4eda4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:19.148549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d725a83-1bf5-4e39-a724-d53eb4eda4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:19.148829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d725a83-1bf5-4e39-a724-d53eb4eda4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:19.149130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d725a83-1bf5-4e39-a724-d53eb4eda4c7 admin admin] PATH_INFO: `/v3/users/5a85cff936184b5d94375fd103493950` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:19.150308 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d725a83-1bf5-4e39-a724-d53eb4eda4c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a85cff936184b5d94375fd103493950)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:19.152397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d725a83-1bf5-4e39-a724-d53eb4eda4c7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:19.367565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7d725a83-1bf5-4e39-a724-d53eb4eda4c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a85cff936184b5d94375fd103493950'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:19.381011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7d725a83-1bf5-4e39-a724-d53eb4eda4c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a85cff936184b5d94375fd103493950'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:19.388858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7d725a83-1bf5-4e39-a724-d53eb4eda4c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a85cff936184b5d94375fd103493950'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:19.450740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7d725a83-1bf5-4e39-a724-d53eb4eda4c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a85cff936184b5d94375fd103493950'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:19.451362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7d725a83-1bf5-4e39-a724-d53eb4eda4c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a85cff936184b5d94375fd103493950'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:19.457880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1171/2367] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:11:19 2022] DELETE /identity/v3/users/5a85cff936184b5d94375fd103493950 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:19.697736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb38f2-b809-48a0-ace4-739839f971e1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:19.709612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb38f2-b809-48a0-ace4-739839f971e1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:19.810481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:19.811174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:19.811448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:19.811712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] PATH_INFO: `/v3/projects/f143f731c6cc4b38ba979827cd390f77` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:19.812815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f143f731c6cc4b38ba979827cd390f77)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:19.814743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:19.917025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:19.928650 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f143f731c6cc4b38ba979827cd390f77', 'request_id': 'req-8388a823-ed6a-4f18-a3b2-31f8d6a22425'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:19.935399 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f143f731c6cc4b38ba979827cd390f77', 'request_id': 'req-8388a823-ed6a-4f18-a3b2-31f8d6a22425'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:19.941285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f143f731c6cc4b38ba979827cd390f77', 'request_id': 'req-8388a823-ed6a-4f18-a3b2-31f8d6a22425'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:20.005313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] The token cache is being invalidate because project f143f731c6cc4b38ba979827cd390f77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:11:20.017771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:20.017771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8388a823-ed6a-4f18-a3b2-31f8d6a22425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:20.017771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1197/2368] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:11:19 2022] DELETE /identity/v3/projects/f143f731c6cc4b38ba979827cd390f77 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:20.018085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa4b7c-27f2-4fbb-81e4-579ab484b486 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:20.018085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa4b7c-27f2-4fbb-81e4-579ab484b486 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:20.018085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa4b7c-27f2-4fbb-81e4-579ab484b486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:20.038723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a6fa4b7c-27f2-4fbb-81e4-579ab484b486 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:20.040837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a6fa4b7c-27f2-4fbb-81e4-579ab484b486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:20.117688 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1172/2369] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:11:20.125934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ae19-d984-483b-8a44-0d911ef27a20 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:20.126225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ae19-d984-483b-8a44-0d911ef27a20 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:20.126642 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ae19-d984-483b-8a44-0d911ef27a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:20.149103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a390ae19-d984-483b-8a44-0d911ef27a20 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:20.150488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a390ae19-d984-483b-8a44-0d911ef27a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:20.160055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1198/2370] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:20.165309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa4b7c-27f2-4fbb-81e4-579ab484b486 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:20.177239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa4b7c-27f2-4fbb-81e4-579ab484b486 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:20.182091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd965-aabf-4f5c-b31d-3ee0279613c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:20.183386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd965-aabf-4f5c-b31d-3ee0279613c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:20.183675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd965-aabf-4f5c-b31d-3ee0279613c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:20.183952 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd965-aabf-4f5c-b31d-3ee0279613c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:20.184877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffd965-aabf-4f5c-b31d-3ee0279613c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:11:20.185195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffd965-aabf-4f5c-b31d-3ee0279613c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:20.186889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffd965-aabf-4f5c-b31d-3ee0279613c6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:20.202923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1173/2371] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:11:20.210302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390ae19-d984-483b-8a44-0d911ef27a20 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:20.220548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390ae19-d984-483b-8a44-0d911ef27a20 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:20.225542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b4410-d34e-427f-be57-7ef93c5d7b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:20.226148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b4410-d34e-427f-be57-7ef93c5d7b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:20.226434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b4410-d34e-427f-be57-7ef93c5d7b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:20.226737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b4410-d34e-427f-be57-7ef93c5d7b0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:20.227369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b4410-d34e-427f-be57-7ef93c5d7b0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:20.229359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b4410-d34e-427f-be57-7ef93c5d7b0b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:20.337746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1199/2372] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:11:20 2022] POST /identity/v3/projects => generated 355 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:20.342583 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd965-aabf-4f5c-b31d-3ee0279613c6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:20.352837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd965-aabf-4f5c-b31d-3ee0279613c6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:20.406537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:20.421427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74dfb1-ab97-4ec9-a837-b62141cb97e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:20.425669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74dfb1-ab97-4ec9-a837-b62141cb97e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:20.425669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74dfb1-ab97-4ec9-a837-b62141cb97e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:20.425669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74dfb1-ab97-4ec9-a837-b62141cb97e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:20.425669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74dfb1-ab97-4ec9-a837-b62141cb97e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:20.427193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74dfb1-ab97-4ec9-a837-b62141cb97e5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:20.548619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b4410-d34e-427f-be57-7ef93c5d7b0b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:20.549547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1174/2373] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:11:20 2022] POST /identity/v3/users => generated 278 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:20.561010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b4410-d34e-427f-be57-7ef93c5d7b0b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:20.565177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7c633-c00c-48ef-b82b-943516b95990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:20.565805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7c633-c00c-48ef-b82b-943516b95990 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:20.566082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7c633-c00c-48ef-b82b-943516b95990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:20.566355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7c633-c00c-48ef-b82b-943516b95990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:20.567075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7c633-c00c-48ef-b82b-943516b95990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:20.568659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7c633-c00c-48ef-b82b-943516b95990 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:20.579564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1200/2374] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:11:20.583547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74dfb1-ab97-4ec9-a837-b62141cb97e5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:20.593709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74dfb1-ab97-4ec9-a837-b62141cb97e5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:20.598460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5043e-133b-410f-97c0-762a626e62d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:20.599087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5043e-133b-410f-97c0-762a626e62d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:20.599364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5043e-133b-410f-97c0-762a626e62d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:20.599642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5043e-133b-410f-97c0-762a626e62d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:20.600284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5043e-133b-410f-97c0-762a626e62d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:20.602007 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5043e-133b-410f-97c0-762a626e62d3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:20.614003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1175/2375] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:11:20.618526 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7c633-c00c-48ef-b82b-943516b95990 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:20.629054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7c633-c00c-48ef-b82b-943516b95990 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:20.633614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9ead7-0649-44d0-9e1f-733f52cb0ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:20.634273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9ead7-0649-44d0-9e1f-733f52cb0ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:20.634559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9ead7-0649-44d0-9e1f-733f52cb0ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:20.634926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9ead7-0649-44d0-9e1f-733f52cb0ddf admin admin] PATH_INFO: `/v3/projects/bbc448e5b2c541ec904a43f3b2aa4bfa/users/4d1076e7e4a9462aa77d7fe5552a6c2a/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:20.650111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9ead7-0649-44d0-9e1f-733f52cb0ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc448e5b2c541ec904a43f3b2aa4bfa, user_id=4d1076e7e4a9462aa77d7fe5552a6c2a, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:20.652794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9ead7-0649-44d0-9e1f-733f52cb0ddf admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:20.807118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1201/2376] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:11:20 2022] PUT /identity/v3/projects/bbc448e5b2c541ec904a43f3b2aa4bfa/users/4d1076e7e4a9462aa77d7fe5552a6c2a/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:11:20.807778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e5e0b-447f-475f-8d05-4f086440c878 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:20.807778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e5e0b-447f-475f-8d05-4f086440c878 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:20.807778 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e5e0b-447f-475f-8d05-4f086440c878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:20.870506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c85e5e0b-447f-475f-8d05-4f086440c878 None None] MFA Rules not processed for user `4d1076e7e4a9462aa77d7fe5552a6c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:20.872749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c85e5e0b-447f-475f-8d05-4f086440c878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:20.968298 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1176/2377] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:20.973234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9ead7-0649-44d0-9e1f-733f52cb0ddf admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:21.065309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9ead7-0649-44d0-9e1f-733f52cb0ddf admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:21.071843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886ca3f-5d63-4dc5-92c4-6170ca3f8f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:21.072691 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886ca3f-5d63-4dc5-92c4-6170ca3f8f86 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:21.072979 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886ca3f-5d63-4dc5-92c4-6170ca3f8f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:21.073258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886ca3f-5d63-4dc5-92c4-6170ca3f8f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:21.073962 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886ca3f-5d63-4dc5-92c4-6170ca3f8f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:21.076816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886ca3f-5d63-4dc5-92c4-6170ca3f8f86 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:21.206810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1202/2378] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:11:20 2022] POST /identity/v3/projects => generated 355 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:21.210937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e5e0b-447f-475f-8d05-4f086440c878 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:21.222887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e5e0b-447f-475f-8d05-4f086440c878 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:21.322226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400509-a885-418b-8c30-1a846894896a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:21.322907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400509-a885-418b-8c30-1a846894896a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:21.323197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400509-a885-418b-8c30-1a846894896a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:21.323457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400509-a885-418b-8c30-1a846894896a admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:21.324232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea400509-a885-418b-8c30-1a846894896a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:21.327687 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea400509-a885-418b-8c30-1a846894896a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:21.419145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1177/2379] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:11:21 2022] POST /identity/v3/users => generated 284 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:21.423539 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886ca3f-5d63-4dc5-92c4-6170ca3f8f86 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:21.440589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886ca3f-5d63-4dc5-92c4-6170ca3f8f86 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:21.449745 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10da60-bc49-4828-a5c6-7d6b1e104859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:21.451001 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10da60-bc49-4828-a5c6-7d6b1e104859 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:21.458869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10da60-bc49-4828-a5c6-7d6b1e104859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:21.458869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10da60-bc49-4828-a5c6-7d6b1e104859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:21.458869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10da60-bc49-4828-a5c6-7d6b1e104859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:21.458869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10da60-bc49-4828-a5c6-7d6b1e104859 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:21.467750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1203/2380] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:11:21.471711 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400509-a885-418b-8c30-1a846894896a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:21.486746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400509-a885-418b-8c30-1a846894896a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:21.501341 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaf126-d745-4124-9d13-98cc911d8f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:21.502529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf126-d745-4124-9d13-98cc911d8f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:21.502911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf126-d745-4124-9d13-98cc911d8f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:21.503477 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf126-d745-4124-9d13-98cc911d8f46 admin admin] PATH_INFO: `/v3/projects/9ea9ae9ce7004cf49f0ac2d7d8065187/users/83f6601a70164dac8a07253d29397362/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:21.542344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afaf126-d745-4124-9d13-98cc911d8f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea9ae9ce7004cf49f0ac2d7d8065187, user_id=83f6601a70164dac8a07253d29397362, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:21.546015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afaf126-d745-4124-9d13-98cc911d8f46 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:21.651184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1178/2381] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:11:21 2022] PUT /identity/v3/projects/9ea9ae9ce7004cf49f0ac2d7d8065187/users/83f6601a70164dac8a07253d29397362/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:11:21.655242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10da60-bc49-4828-a5c6-7d6b1e104859 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:21.665559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10da60-bc49-4828-a5c6-7d6b1e104859 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:21.705814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:21.747468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54d73a-c21e-428e-9f14-365ccafbc23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:21.748125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54d73a-c21e-428e-9f14-365ccafbc23b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:21.748433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54d73a-c21e-428e-9f14-365ccafbc23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:21.748733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54d73a-c21e-428e-9f14-365ccafbc23b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:21.749385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54d73a-c21e-428e-9f14-365ccafbc23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:21.751056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54d73a-c21e-428e-9f14-365ccafbc23b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:21.761728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1204/2382] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:11:21.766237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaf126-d745-4124-9d13-98cc911d8f46 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:21.776997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaf126-d745-4124-9d13-98cc911d8f46 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:21.781867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1f626-6aed-4687-8443-1f0baaafa039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:21.782521 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f626-6aed-4687-8443-1f0baaafa039 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:21.782835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f626-6aed-4687-8443-1f0baaafa039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:21.783110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f626-6aed-4687-8443-1f0baaafa039 admin admin] PATH_INFO: `/v3/projects/9ea9ae9ce7004cf49f0ac2d7d8065187/users/83f6601a70164dac8a07253d29397362/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:21.784992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1f626-6aed-4687-8443-1f0baaafa039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea9ae9ce7004cf49f0ac2d7d8065187, user_id=83f6601a70164dac8a07253d29397362, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:21.787775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1f626-6aed-4687-8443-1f0baaafa039 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:21.799744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1179/2383] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:11:21 2022] PUT /identity/v3/projects/9ea9ae9ce7004cf49f0ac2d7d8065187/users/83f6601a70164dac8a07253d29397362/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:21.810621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39482dd8-f575-48a3-aa65-c1b1f31cf706 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:21.811536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39482dd8-f575-48a3-aa65-c1b1f31cf706 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:21.811820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39482dd8-f575-48a3-aa65-c1b1f31cf706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:21.842638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-39482dd8-f575-48a3-aa65-c1b1f31cf706 None None] MFA Rules not processed for user `83f6601a70164dac8a07253d29397362`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:21.844192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-39482dd8-f575-48a3-aa65-c1b1f31cf706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:21.916381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1205/2384] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:21.935990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1f626-6aed-4687-8443-1f0baaafa039 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:21.937791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e4d1f626-6aed-4687-8443-1f0baaafa039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:21.948954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:22.002119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1f626-6aed-4687-8443-1f0baaafa039 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:22.007571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80a3b0-5c67-4f9a-84b8-b495dfc62d2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:22.008190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80a3b0-5c67-4f9a-84b8-b495dfc62d2c service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:22.008430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80a3b0-5c67-4f9a-84b8-b495dfc62d2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:22.008704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80a3b0-5c67-4f9a-84b8-b495dfc62d2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:22.031599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80a3b0-5c67-4f9a-84b8-b495dfc62d2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:22.034036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80a3b0-5c67-4f9a-84b8-b495dfc62d2c service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:22.048664 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1180/2385] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:22.285560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39482dd8-f575-48a3-aa65-c1b1f31cf706 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:22.287365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-39482dd8-f575-48a3-aa65-c1b1f31cf706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:22.306283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39482dd8-f575-48a3-aa65-c1b1f31cf706 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:22.313116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1e35f-9f9b-4ac1-a51f-cc78dfb1d763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:22.313116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1e35f-9f9b-4ac1-a51f-cc78dfb1d763 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:22.313116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1e35f-9f9b-4ac1-a51f-cc78dfb1d763 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:22.313116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1e35f-9f9b-4ac1-a51f-cc78dfb1d763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:22.380731 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1e35f-9f9b-4ac1-a51f-cc78dfb1d763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:22.382279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1e35f-9f9b-4ac1-a51f-cc78dfb1d763 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:22.392574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1206/2386] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:32.281038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=22 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:32.388206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=76 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:41.053231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80a3b0-5c67-4f9a-84b8-b495dfc62d2c service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:41.114333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80a3b0-5c67-4f9a-84b8-b495dfc62d2c service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:41.118948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:41.119568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:41.119850 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:41.120121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] PATH_INFO: `/v3/users/4d1076e7e4a9462aa77d7fe5552a6c2a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:41.121235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1076e7e4a9462aa77d7fe5552a6c2a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:41.123186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:41.350441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1076e7e4a9462aa77d7fe5552a6c2a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:41.362584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1076e7e4a9462aa77d7fe5552a6c2a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:41.369495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1076e7e4a9462aa77d7fe5552a6c2a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:41.450310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1076e7e4a9462aa77d7fe5552a6c2a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:41.450956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1076e7e4a9462aa77d7fe5552a6c2a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:41.457643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1181/2387] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:11:41 2022] DELETE /identity/v3/users/4d1076e7e4a9462aa77d7fe5552a6c2a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:41.462099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e7299-8bf1-4c21-92a2-0c364119a56f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:41.462577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e7299-8bf1-4c21-92a2-0c364119a56f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:41.462772 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e7299-8bf1-4c21-92a2-0c364119a56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:41.482646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8d4e7299-8bf1-4c21-92a2-0c364119a56f None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:41.483892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8d4e7299-8bf1-4c21-92a2-0c364119a56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:41.542614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1207/2388] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:41.552992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:41.554411 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:41.612354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec69852-116a-4b30-ba61-b84d53e01aa8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:41.615912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d9b40-4eac-48ca-8639-0d7c4f9009c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:41.616515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d9b40-4eac-48ca-8639-0d7c4f9009c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:41.616798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d9b40-4eac-48ca-8639-0d7c4f9009c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:41.617068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d9b40-4eac-48ca-8639-0d7c4f9009c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:41.626427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d9b40-4eac-48ca-8639-0d7c4f9009c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:41.627905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d9b40-4eac-48ca-8639-0d7c4f9009c1 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:41.637130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1182/2389] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:11:42.073488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e7299-8bf1-4c21-92a2-0c364119a56f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:42.075034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8d4e7299-8bf1-4c21-92a2-0c364119a56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:42.087075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e7299-8bf1-4c21-92a2-0c364119a56f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:42.091670 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:42.092787 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:42.093193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:42.093636 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] PATH_INFO: `/v3/projects/bbc448e5b2c541ec904a43f3b2aa4bfa` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:42.094966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbc448e5b2c541ec904a43f3b2aa4bfa)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:42.096885 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:42.210529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbc448e5b2c541ec904a43f3b2aa4bfa', 'request_id': 'req-dc0058e1-4975-42b6-8f13-fddb5571d40f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:42.216892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbc448e5b2c541ec904a43f3b2aa4bfa', 'request_id': 'req-dc0058e1-4975-42b6-8f13-fddb5571d40f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:42.223071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbc448e5b2c541ec904a43f3b2aa4bfa', 'request_id': 'req-dc0058e1-4975-42b6-8f13-fddb5571d40f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:42.336001 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] The token cache is being invalidate because project bbc448e5b2c541ec904a43f3b2aa4bfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:11:42.336226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:42.336860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:42.338312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1208/2390] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:11:42 2022] DELETE /identity/v3/projects/bbc448e5b2c541ec904a43f3b2aa4bfa => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:42.342306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d9b40-4eac-48ca-8639-0d7c4f9009c1 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:42.352657 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d9b40-4eac-48ca-8639-0d7c4f9009c1 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:42.370403 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:42.409250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fc968-17d3-4406-a478-dc08b016f321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:42.409884 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fc968-17d3-4406-a478-dc08b016f321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:42.410168 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fc968-17d3-4406-a478-dc08b016f321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:42.410453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fc968-17d3-4406-a478-dc08b016f321 admin admin] PATH_INFO: `/v3/users/83f6601a70164dac8a07253d29397362` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:42.411628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fc968-17d3-4406-a478-dc08b016f321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f6601a70164dac8a07253d29397362)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:42.413566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fc968-17d3-4406-a478-dc08b016f321 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:42.602429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c3fc968-17d3-4406-a478-dc08b016f321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f6601a70164dac8a07253d29397362'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:42.614559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c3fc968-17d3-4406-a478-dc08b016f321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f6601a70164dac8a07253d29397362'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:42.621458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c3fc968-17d3-4406-a478-dc08b016f321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f6601a70164dac8a07253d29397362'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:42.696550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c3fc968-17d3-4406-a478-dc08b016f321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f6601a70164dac8a07253d29397362'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:42.697173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c3fc968-17d3-4406-a478-dc08b016f321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f6601a70164dac8a07253d29397362'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:42.703872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1183/2391] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:11:42 2022] DELETE /identity/v3/users/83f6601a70164dac8a07253d29397362 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:43.081009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:43.082502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:43.095405 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0058e1-4975-42b6-8f13-fddb5571d40f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:43.152863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:43.153435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:43.153703 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:43.153977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] PATH_INFO: `/v3/projects/9ea9ae9ce7004cf49f0ac2d7d8065187` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:43.155426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea9ae9ce7004cf49f0ac2d7d8065187)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:43.157332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:43.229698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:43.245907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea9ae9ce7004cf49f0ac2d7d8065187', 'request_id': 'req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:43.252704 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea9ae9ce7004cf49f0ac2d7d8065187', 'request_id': 'req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:43.258231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea9ae9ce7004cf49f0ac2d7d8065187', 'request_id': 'req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:43.329851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] The token cache is being invalidate because project 9ea9ae9ce7004cf49f0ac2d7d8065187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:11:43.330135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:43.330809 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9eaa2b5-ff33-4c2a-bf93-6ca3b8af0c29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:43.332303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1209/2392] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:11:43 2022] DELETE /identity/v3/projects/9ea9ae9ce7004cf49f0ac2d7d8065187 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:43.337101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea18fc7-f339-4836-9ed0-1d1d39abda6b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:43.337671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea18fc7-f339-4836-9ed0-1d1d39abda6b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:43.337671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea18fc7-f339-4836-9ed0-1d1d39abda6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:43.358903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8ea18fc7-f339-4836-9ed0-1d1d39abda6b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:43.360145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8ea18fc7-f339-4836-9ed0-1d1d39abda6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:43.418658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1184/2393] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:43.428363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee435d-817c-4e6a-bc13-57b6059b1e1c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:43.428665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee435d-817c-4e6a-bc13-57b6059b1e1c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:43.428941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee435d-817c-4e6a-bc13-57b6059b1e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:43.449673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2dee435d-817c-4e6a-bc13-57b6059b1e1c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:43.450936 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2dee435d-817c-4e6a-bc13-57b6059b1e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:43.459081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1210/2394] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:43.463227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea18fc7-f339-4836-9ed0-1d1d39abda6b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:43.473608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea18fc7-f339-4836-9ed0-1d1d39abda6b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:43.478338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a14a3-c4c3-4b42-a120-89832c543b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:43.479007 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a14a3-c4c3-4b42-a120-89832c543b00 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:43.479295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a14a3-c4c3-4b42-a120-89832c543b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:43.481122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a14a3-c4c3-4b42-a120-89832c543b00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:43.481122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a14a3-c4c3-4b42-a120-89832c543b00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:11:43.481122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a14a3-c4c3-4b42-a120-89832c543b00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:43.482235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a14a3-c4c3-4b42-a120-89832c543b00 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:43.494490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1185/2395] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:11:43.503797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee435d-817c-4e6a-bc13-57b6059b1e1c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:43.513226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee435d-817c-4e6a-bc13-57b6059b1e1c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:43.518164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a7fa6-3e95-4af3-aeca-67180df0ba84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:43.521347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a7fa6-3e95-4af3-aeca-67180df0ba84 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:43.521347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a7fa6-3e95-4af3-aeca-67180df0ba84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:43.521347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a7fa6-3e95-4af3-aeca-67180df0ba84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:43.521347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a7fa6-3e95-4af3-aeca-67180df0ba84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:43.521899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a7fa6-3e95-4af3-aeca-67180df0ba84 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:43.628006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1211/2396] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:11:43 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:43.632099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a14a3-c4c3-4b42-a120-89832c543b00 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:43.642203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a14a3-c4c3-4b42-a120-89832c543b00 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:43.669123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:43.699027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8995ad-a989-48c1-aee2-6da562893501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:43.699698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8995ad-a989-48c1-aee2-6da562893501 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:43.699977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8995ad-a989-48c1-aee2-6da562893501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:43.700252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8995ad-a989-48c1-aee2-6da562893501 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:43.700913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8995ad-a989-48c1-aee2-6da562893501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:43.702878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8995ad-a989-48c1-aee2-6da562893501 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:43.809759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1186/2397] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:11:43 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:43.813923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a7fa6-3e95-4af3-aeca-67180df0ba84 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:43.824447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a7fa6-3e95-4af3-aeca-67180df0ba84 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:43.829717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19416f-3f06-4d60-a9a7-cfdadde21273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:43.830329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19416f-3f06-4d60-a9a7-cfdadde21273 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:43.830612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19416f-3f06-4d60-a9a7-cfdadde21273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:43.830919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19416f-3f06-4d60-a9a7-cfdadde21273 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:43.831728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19416f-3f06-4d60-a9a7-cfdadde21273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:43.833506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19416f-3f06-4d60-a9a7-cfdadde21273 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:43.848109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1212/2398] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:11:43.849244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8995ad-a989-48c1-aee2-6da562893501 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:43.860077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8995ad-a989-48c1-aee2-6da562893501 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:43.865450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922078c-6981-4e76-bdcb-0e17e6e826e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:43.866084 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922078c-6981-4e76-bdcb-0e17e6e826e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:43.866371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922078c-6981-4e76-bdcb-0e17e6e826e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:43.866652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922078c-6981-4e76-bdcb-0e17e6e826e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:43.867327 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922078c-6981-4e76-bdcb-0e17e6e826e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:43.868922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922078c-6981-4e76-bdcb-0e17e6e826e2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:43.879842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1187/2399] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:11:43.884013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19416f-3f06-4d60-a9a7-cfdadde21273 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:43.896323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19416f-3f06-4d60-a9a7-cfdadde21273 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:43.901010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b4040-82ac-41ad-828d-6f135c9c3dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:43.901655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b4040-82ac-41ad-828d-6f135c9c3dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:43.901933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b4040-82ac-41ad-828d-6f135c9c3dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:43.902204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b4040-82ac-41ad-828d-6f135c9c3dc8 admin admin] PATH_INFO: `/v3/projects/fd8bdf863fce4b96854ff8137f73f8c9/users/da532a5bc9e0492bbb8023c6ca012fc7/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:43.917913 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b4040-82ac-41ad-828d-6f135c9c3dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8bdf863fce4b96854ff8137f73f8c9, user_id=da532a5bc9e0492bbb8023c6ca012fc7, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:43.926393 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b4040-82ac-41ad-828d-6f135c9c3dc8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:44.005316 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1213/2400] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:11:43 2022] PUT /identity/v3/projects/fd8bdf863fce4b96854ff8137f73f8c9/users/da532a5bc9e0492bbb8023c6ca012fc7/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:44.016293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bf496-295c-4bec-9297-9bd78634d79d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:44.016578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bf496-295c-4bec-9297-9bd78634d79d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:44.016879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bf496-295c-4bec-9297-9bd78634d79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:44.045321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-454bf496-295c-4bec-9297-9bd78634d79d None None] MFA Rules not processed for user `da532a5bc9e0492bbb8023c6ca012fc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:44.048565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-454bf496-295c-4bec-9297-9bd78634d79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:44.108853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1188/2401] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:44.113268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b4040-82ac-41ad-828d-6f135c9c3dc8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:44.173624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b4040-82ac-41ad-828d-6f135c9c3dc8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:44.178301 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc4170-f71e-4f91-bbc3-7c472ca3ea1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:44.178882 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4170-f71e-4f91-bbc3-7c472ca3ea1b admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:44.179166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4170-f71e-4f91-bbc3-7c472ca3ea1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:44.179440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4170-f71e-4f91-bbc3-7c472ca3ea1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:44.180065 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc4170-f71e-4f91-bbc3-7c472ca3ea1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:44.181960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc4170-f71e-4f91-bbc3-7c472ca3ea1b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:44.303860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1214/2402] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:11:44 2022] POST /identity/v3/projects => generated 367 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:44.307655 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bf496-295c-4bec-9297-9bd78634d79d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:44.318809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bf496-295c-4bec-9297-9bd78634d79d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:44.374897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c5410-66e4-4c23-b80a-b2681ad3c9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:44.375591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c5410-66e4-4c23-b80a-b2681ad3c9ff admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:44.375747 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c5410-66e4-4c23-b80a-b2681ad3c9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:44.376017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c5410-66e4-4c23-b80a-b2681ad3c9ff admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:44.376652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c5410-66e4-4c23-b80a-b2681ad3c9ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:44.378593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c5410-66e4-4c23-b80a-b2681ad3c9ff admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:44.478229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1189/2403] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:11:44 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:44.482187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc4170-f71e-4f91-bbc3-7c472ca3ea1b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:44.493379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc4170-f71e-4f91-bbc3-7c472ca3ea1b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:44.498274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1166d-7d65-4e6a-a16f-2f4a79c63b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:44.498927 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1166d-7d65-4e6a-a16f-2f4a79c63b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:44.499216 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1166d-7d65-4e6a-a16f-2f4a79c63b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:44.499519 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1166d-7d65-4e6a-a16f-2f4a79c63b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:44.500693 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1166d-7d65-4e6a-a16f-2f4a79c63b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:44.502328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1166d-7d65-4e6a-a16f-2f4a79c63b3b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:44.513480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1215/2404] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:11:44.518086 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c5410-66e4-4c23-b80a-b2681ad3c9ff admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:44.528158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c5410-66e4-4c23-b80a-b2681ad3c9ff admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:44.533136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef877aa-1692-4c9a-bb31-2bc52c7b546e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:44.533782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef877aa-1692-4c9a-bb31-2bc52c7b546e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:44.534075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef877aa-1692-4c9a-bb31-2bc52c7b546e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:44.534345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef877aa-1692-4c9a-bb31-2bc52c7b546e admin admin] PATH_INFO: `/v3/projects/8e1f9359a3a4429c9f16d1bfe82a5bbe/users/473bfab518ec4ee584a3ea0aa74f31d1/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:44.549375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef877aa-1692-4c9a-bb31-2bc52c7b546e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1f9359a3a4429c9f16d1bfe82a5bbe, user_id=473bfab518ec4ee584a3ea0aa74f31d1, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:44.552072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef877aa-1692-4c9a-bb31-2bc52c7b546e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:44.639296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1190/2405] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:11:44 2022] PUT /identity/v3/projects/8e1f9359a3a4429c9f16d1bfe82a5bbe/users/473bfab518ec4ee584a3ea0aa74f31d1/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:44.643391 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1166d-7d65-4e6a-a16f-2f4a79c63b3b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:44.653218 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1166d-7d65-4e6a-a16f-2f4a79c63b3b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:44.673496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:44.709262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bef755-0bfb-4f94-803d-13607aff8fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:44.709861 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef755-0bfb-4f94-803d-13607aff8fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:44.710138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef755-0bfb-4f94-803d-13607aff8fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:44.710410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef755-0bfb-4f94-803d-13607aff8fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:44.711231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bef755-0bfb-4f94-803d-13607aff8fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:44.712843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bef755-0bfb-4f94-803d-13607aff8fb4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:44.722923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1216/2406] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:11:44.727058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef877aa-1692-4c9a-bb31-2bc52c7b546e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:44.732624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:44.738398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef877aa-1692-4c9a-bb31-2bc52c7b546e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:44.743063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f02285-ed6b-4013-9576-bb61b6333b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:44.743713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f02285-ed6b-4013-9576-bb61b6333b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:44.743989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f02285-ed6b-4013-9576-bb61b6333b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:44.744260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f02285-ed6b-4013-9576-bb61b6333b31 admin admin] PATH_INFO: `/v3/projects/8e1f9359a3a4429c9f16d1bfe82a5bbe/users/473bfab518ec4ee584a3ea0aa74f31d1/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:44.746137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f02285-ed6b-4013-9576-bb61b6333b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1f9359a3a4429c9f16d1bfe82a5bbe, user_id=473bfab518ec4ee584a3ea0aa74f31d1, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:44.748821 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f02285-ed6b-4013-9576-bb61b6333b31 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:44.759660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1191/2407] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:11:44 2022] PUT /identity/v3/projects/8e1f9359a3a4429c9f16d1bfe82a5bbe/users/473bfab518ec4ee584a3ea0aa74f31d1/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:44.769972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ccce7-00dc-489b-bb65-753c6079d253 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:44.770262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ccce7-00dc-489b-bb65-753c6079d253 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:44.770541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ccce7-00dc-489b-bb65-753c6079d253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:44.798658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-052ccce7-00dc-489b-bb65-753c6079d253 None None] MFA Rules not processed for user `473bfab518ec4ee584a3ea0aa74f31d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:44.799891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-052ccce7-00dc-489b-bb65-753c6079d253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:44.872810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1217/2408] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:44.884455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f02285-ed6b-4013-9576-bb61b6333b31 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:44.886111 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-23f02285-ed6b-4013-9576-bb61b6333b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:44.944003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f02285-ed6b-4013-9576-bb61b6333b31 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:44.947680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e924b89-4715-4f89-bd63-31ef3db75a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:44.948282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e924b89-4715-4f89-bd63-31ef3db75a66 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:44.948562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e924b89-4715-4f89-bd63-31ef3db75a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:44.948834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e924b89-4715-4f89-bd63-31ef3db75a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:44.957622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e924b89-4715-4f89-bd63-31ef3db75a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:44.959124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e924b89-4715-4f89-bd63-31ef3db75a66 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:44.968483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1192/2409] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:45.167912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ccce7-00dc-489b-bb65-753c6079d253 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:45.178325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ccce7-00dc-489b-bb65-753c6079d253 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:45.181995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51271c-b5e7-48bf-b148-00b4462fc286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:45.182594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51271c-b5e7-48bf-b148-00b4462fc286 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:45.182901 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51271c-b5e7-48bf-b148-00b4462fc286 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:45.183177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51271c-b5e7-48bf-b148-00b4462fc286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:45.243228 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51271c-b5e7-48bf-b148-00b4462fc286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:45.244773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51271c-b5e7-48bf-b148-00b4462fc286 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:45.253784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1218/2410] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:51.656624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e924b89-4715-4f89-bd63-31ef3db75a66 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:51.762037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e924b89-4715-4f89-bd63-31ef3db75a66 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:51.766563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:51.767210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:51.767490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:51.767764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] PATH_INFO: `/v3/users/da532a5bc9e0492bbb8023c6ca012fc7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:51.768886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da532a5bc9e0492bbb8023c6ca012fc7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:51.770847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:51.968602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da532a5bc9e0492bbb8023c6ca012fc7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:51.973941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:51.981926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da532a5bc9e0492bbb8023c6ca012fc7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:51.988767 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da532a5bc9e0492bbb8023c6ca012fc7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:52.073425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da532a5bc9e0492bbb8023c6ca012fc7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:52.074036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da532a5bc9e0492bbb8023c6ca012fc7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:52.080085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1193/2411] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:11:51 2022] DELETE /identity/v3/users/da532a5bc9e0492bbb8023c6ca012fc7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:52.085083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772ffd0-c993-4b7b-8d53-f3edce7fb738 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:52.085569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772ffd0-c993-4b7b-8d53-f3edce7fb738 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:52.085669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772ffd0-c993-4b7b-8d53-f3edce7fb738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:52.133754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e772ffd0-c993-4b7b-8d53-f3edce7fb738 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:52.135236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e772ffd0-c993-4b7b-8d53-f3edce7fb738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:52.147295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:52.198114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1219/2412] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:52.208325 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:52.209816 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:52.301643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10fefa-70bd-4a8b-97ba-b4246be4a7c6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:52.305215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957a8c7-ef79-4d62-878a-fa38adc18e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:52.305831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957a8c7-ef79-4d62-878a-fa38adc18e26 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:52.306104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957a8c7-ef79-4d62-878a-fa38adc18e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:52.306379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957a8c7-ef79-4d62-878a-fa38adc18e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:52.316490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957a8c7-ef79-4d62-878a-fa38adc18e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:52.317047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957a8c7-ef79-4d62-878a-fa38adc18e26 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:52.326210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1194/2413] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:52.756319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772ffd0-c993-4b7b-8d53-f3edce7fb738 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:52.757976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e772ffd0-c993-4b7b-8d53-f3edce7fb738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:52.771570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772ffd0-c993-4b7b-8d53-f3edce7fb738 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:52.776578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:52.777266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:52.777552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:52.777832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] PATH_INFO: `/v3/projects/fd8bdf863fce4b96854ff8137f73f8c9` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:52.779331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd8bdf863fce4b96854ff8137f73f8c9)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:52.781286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:52.896783 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd8bdf863fce4b96854ff8137f73f8c9', 'request_id': 'req-b9c8a0e4-008f-44f0-bb62-5b64319a813a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:52.902940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd8bdf863fce4b96854ff8137f73f8c9', 'request_id': 'req-b9c8a0e4-008f-44f0-bb62-5b64319a813a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:52.908824 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd8bdf863fce4b96854ff8137f73f8c9', 'request_id': 'req-b9c8a0e4-008f-44f0-bb62-5b64319a813a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:52.991883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] The token cache is being invalidate because project fd8bdf863fce4b96854ff8137f73f8c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:11:52.992172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:52.992788 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:52.994253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1220/2414] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:11:52 2022] DELETE /identity/v3/projects/fd8bdf863fce4b96854ff8137f73f8c9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:52.998481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957a8c7-ef79-4d62-878a-fa38adc18e26 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:53.008995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957a8c7-ef79-4d62-878a-fa38adc18e26 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:53.063645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b193d-c0f0-4878-9d54-f8c77e4bc62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:53.064273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b193d-c0f0-4878-9d54-f8c77e4bc62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:53.064549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b193d-c0f0-4878-9d54-f8c77e4bc62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:53.065479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b193d-c0f0-4878-9d54-f8c77e4bc62f admin admin] PATH_INFO: `/v3/users/473bfab518ec4ee584a3ea0aa74f31d1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:53.066561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b193d-c0f0-4878-9d54-f8c77e4bc62f admin admin] RBAC: Authorizing `identity:delete_user(user_id=473bfab518ec4ee584a3ea0aa74f31d1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:53.068540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b193d-c0f0-4878-9d54-f8c77e4bc62f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:53.305264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:53.311092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-768b193d-c0f0-4878-9d54-f8c77e4bc62f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '473bfab518ec4ee584a3ea0aa74f31d1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:53.322671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-768b193d-c0f0-4878-9d54-f8c77e4bc62f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '473bfab518ec4ee584a3ea0aa74f31d1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:53.329803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-768b193d-c0f0-4878-9d54-f8c77e4bc62f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '473bfab518ec4ee584a3ea0aa74f31d1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:53.409849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-768b193d-c0f0-4878-9d54-f8c77e4bc62f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '473bfab518ec4ee584a3ea0aa74f31d1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:53.410478 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-768b193d-c0f0-4878-9d54-f8c77e4bc62f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '473bfab518ec4ee584a3ea0aa74f31d1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:53.416629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1195/2415] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:11:52 2022] DELETE /identity/v3/users/473bfab518ec4ee584a3ea0aa74f31d1 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:11:53.800940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:53.802460 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:53.816557 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8a0e4-008f-44f0-bb62-5b64319a813a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:53.873155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:53.873793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:53.874075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:53.874352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] PATH_INFO: `/v3/projects/8e1f9359a3a4429c9f16d1bfe82a5bbe` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:53.875541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e1f9359a3a4429c9f16d1bfe82a5bbe)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:53.877958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:53.956137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:53.959739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e1f9359a3a4429c9f16d1bfe82a5bbe', 'request_id': 'req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:53.965831 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e1f9359a3a4429c9f16d1bfe82a5bbe', 'request_id': 'req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:53.971470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e1f9359a3a4429c9f16d1bfe82a5bbe', 'request_id': 'req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:54.059944 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] The token cache is being invalidate because project 8e1f9359a3a4429c9f16d1bfe82a5bbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:11:54.060226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:54.060836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f6a7ac32-e44b-4196-b096-07e7bd10cfc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:54.062387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1221/2416] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:11:53 2022] DELETE /identity/v3/projects/8e1f9359a3a4429c9f16d1bfe82a5bbe => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:54.067854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b694d-a3c2-49ee-9f88-d2d558f6d9b4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:54.068359 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b694d-a3c2-49ee-9f88-d2d558f6d9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:54.068466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b694d-a3c2-49ee-9f88-d2d558f6d9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:54.116105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-240b694d-a3c2-49ee-9f88-d2d558f6d9b4 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:54.117305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-240b694d-a3c2-49ee-9f88-d2d558f6d9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:54.174495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1196/2417] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:54.182251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845840f-e9cf-40f2-abf6-2e4a22b0a4ed None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:54.182532 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845840f-e9cf-40f2-abf6-2e4a22b0a4ed None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:54.182839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845840f-e9cf-40f2-abf6-2e4a22b0a4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:54.202741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a845840f-e9cf-40f2-abf6-2e4a22b0a4ed None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:54.203934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a845840f-e9cf-40f2-abf6-2e4a22b0a4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:54.213423 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1222/2418] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:54.216362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b694d-a3c2-49ee-9f88-d2d558f6d9b4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:54.225972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b694d-a3c2-49ee-9f88-d2d558f6d9b4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:54.230580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719195a3-a6ff-47c7-969a-7bc2d334b470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:54.231234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719195a3-a6ff-47c7-969a-7bc2d334b470 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:54.231517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719195a3-a6ff-47c7-969a-7bc2d334b470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:54.231794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719195a3-a6ff-47c7-969a-7bc2d334b470 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:54.232467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719195a3-a6ff-47c7-969a-7bc2d334b470 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:11:54.232774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719195a3-a6ff-47c7-969a-7bc2d334b470 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:54.234372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719195a3-a6ff-47c7-969a-7bc2d334b470 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:54.246351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1197/2419] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:11:54.250445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845840f-e9cf-40f2-abf6-2e4a22b0a4ed None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:54.260744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845840f-e9cf-40f2-abf6-2e4a22b0a4ed None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:54.265386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6f481-e75d-4f78-9cf4-a00d36bf2fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:54.265987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6f481-e75d-4f78-9cf4-a00d36bf2fda admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:54.266261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6f481-e75d-4f78-9cf4-a00d36bf2fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:54.266532 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6f481-e75d-4f78-9cf4-a00d36bf2fda admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:54.267190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6f481-e75d-4f78-9cf4-a00d36bf2fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:54.269070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6f481-e75d-4f78-9cf4-a00d36bf2fda admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:54.397961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1223/2420] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:11:54 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:54.399102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719195a3-a6ff-47c7-969a-7bc2d334b470 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:54.410025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719195a3-a6ff-47c7-969a-7bc2d334b470 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:54.466593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3484f92-201c-46d6-9456-c7ca967f57ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:54.467148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3484f92-201c-46d6-9456-c7ca967f57ae admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:54.467424 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3484f92-201c-46d6-9456-c7ca967f57ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:54.467690 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3484f92-201c-46d6-9456-c7ca967f57ae admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:54.468319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3484f92-201c-46d6-9456-c7ca967f57ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:54.470216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3484f92-201c-46d6-9456-c7ca967f57ae admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:54.542088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1198/2421] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:11:54 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:11:54.546237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6f481-e75d-4f78-9cf4-a00d36bf2fda admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:54.557417 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6f481-e75d-4f78-9cf4-a00d36bf2fda admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:54.562081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604e677-4cde-4059-ba1d-e4d6e402498d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:54.562769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604e677-4cde-4059-ba1d-e4d6e402498d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:54.563053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604e677-4cde-4059-ba1d-e4d6e402498d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:54.563328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604e677-4cde-4059-ba1d-e4d6e402498d admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:54.563997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604e677-4cde-4059-ba1d-e4d6e402498d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:54.566255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604e677-4cde-4059-ba1d-e4d6e402498d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:54.576226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1224/2422] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:11:54.584652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3484f92-201c-46d6-9456-c7ca967f57ae admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:54.594766 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3484f92-201c-46d6-9456-c7ca967f57ae admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:54.599921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a64104-61de-4899-a236-62eeb3e92010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:54.600518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a64104-61de-4899-a236-62eeb3e92010 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:54.600800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a64104-61de-4899-a236-62eeb3e92010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:54.601076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a64104-61de-4899-a236-62eeb3e92010 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:54.601717 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a64104-61de-4899-a236-62eeb3e92010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:54.603366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a64104-61de-4899-a236-62eeb3e92010 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:54.605384 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:54.614970 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1199/2423] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:11:54.618799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604e677-4cde-4059-ba1d-e4d6e402498d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:54.628960 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604e677-4cde-4059-ba1d-e4d6e402498d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:54.633698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc224587-aa3f-41f7-8f01-8b592cced0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:54.634348 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc224587-aa3f-41f7-8f01-8b592cced0f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:54.634632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc224587-aa3f-41f7-8f01-8b592cced0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:54.634937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc224587-aa3f-41f7-8f01-8b592cced0f2 admin admin] PATH_INFO: `/v3/projects/2a64e4583fa44c2c8a9ba72742fdd8ac/users/2f744c06f537495ebc1d629db18bf4c9/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:54.649384 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc224587-aa3f-41f7-8f01-8b592cced0f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a64e4583fa44c2c8a9ba72742fdd8ac, user_id=2f744c06f537495ebc1d629db18bf4c9, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:54.652085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc224587-aa3f-41f7-8f01-8b592cced0f2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:54.742986 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1225/2424] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:11:54 2022] PUT /identity/v3/projects/2a64e4583fa44c2c8a9ba72742fdd8ac/users/2f744c06f537495ebc1d629db18bf4c9/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:54.753335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cc5b7-758a-40be-8869-a1422d3faa24 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:54.753628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cc5b7-758a-40be-8869-a1422d3faa24 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:54.759982 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cc5b7-758a-40be-8869-a1422d3faa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:54.781806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-338cc5b7-758a-40be-8869-a1422d3faa24 None None] MFA Rules not processed for user `2f744c06f537495ebc1d629db18bf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:54.783093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-338cc5b7-758a-40be-8869-a1422d3faa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:54.851495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1200/2425] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:54.863388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc224587-aa3f-41f7-8f01-8b592cced0f2 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:54.865159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cc224587-aa3f-41f7-8f01-8b592cced0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:54.941942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc224587-aa3f-41f7-8f01-8b592cced0f2 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:54.945606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35b0dd-5b57-4fc2-a579-e7f3669c5bf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:54.946206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35b0dd-5b57-4fc2-a579-e7f3669c5bf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:54.946485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35b0dd-5b57-4fc2-a579-e7f3669c5bf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:54.946786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35b0dd-5b57-4fc2-a579-e7f3669c5bf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:54.955101 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35b0dd-5b57-4fc2-a579-e7f3669c5bf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:54.957053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35b0dd-5b57-4fc2-a579-e7f3669c5bf0 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:54.965780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1226/2426] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:11:54.985015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cc5b7-758a-40be-8869-a1422d3faa24 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:55.045769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cc5b7-758a-40be-8869-a1422d3faa24 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:55.050300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:55.050946 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:55.051221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:55.051490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] PATH_INFO: `/v3/users/2f744c06f537495ebc1d629db18bf4c9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:55.052599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f744c06f537495ebc1d629db18bf4c9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:55.054526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:55.207055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f744c06f537495ebc1d629db18bf4c9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:55.218871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f744c06f537495ebc1d629db18bf4c9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:55.226260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f744c06f537495ebc1d629db18bf4c9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:55.290210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f744c06f537495ebc1d629db18bf4c9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:55.290843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f744c06f537495ebc1d629db18bf4c9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:55.297073 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1201/2427] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:11:54 2022] DELETE /identity/v3/users/2f744c06f537495ebc1d629db18bf4c9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:55.302026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00baaf-e92c-4c29-ab3a-aa08b73ac376 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:55.302309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00baaf-e92c-4c29-ab3a-aa08b73ac376 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:55.302581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00baaf-e92c-4c29-ab3a-aa08b73ac376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:55.321806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1f00baaf-e92c-4c29-ab3a-aa08b73ac376 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:55.323024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1f00baaf-e92c-4c29-ab3a-aa08b73ac376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:55.360879 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:55.381368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1227/2428] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:55.390487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:55.391904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:55.401880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:55.450354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196cafa-d664-456b-8754-45d1e2024b4b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:55.453980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745d2b0-b999-4d71-ac5e-37f4eeac4705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:55.457473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745d2b0-b999-4d71-ac5e-37f4eeac4705 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:55.457473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745d2b0-b999-4d71-ac5e-37f4eeac4705 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:55.457473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745d2b0-b999-4d71-ac5e-37f4eeac4705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:55.464500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745d2b0-b999-4d71-ac5e-37f4eeac4705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:55.465944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745d2b0-b999-4d71-ac5e-37f4eeac4705 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:55.476813 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1202/2429] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:56.252802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00baaf-e92c-4c29-ab3a-aa08b73ac376 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:56.254449 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1f00baaf-e92c-4c29-ab3a-aa08b73ac376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:56.267692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00baaf-e92c-4c29-ab3a-aa08b73ac376 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:56.272814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:56.273682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:56.274139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:56.274570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] PATH_INFO: `/v3/projects/2a64e4583fa44c2c8a9ba72742fdd8ac` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:56.275950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a64e4583fa44c2c8a9ba72742fdd8ac)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:56.278257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:56.389499 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a64e4583fa44c2c8a9ba72742fdd8ac', 'request_id': 'req-0c79ade4-aa78-4524-9824-ff09c2a4860e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:56.396021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a64e4583fa44c2c8a9ba72742fdd8ac', 'request_id': 'req-0c79ade4-aa78-4524-9824-ff09c2a4860e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:56.402196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a64e4583fa44c2c8a9ba72742fdd8ac', 'request_id': 'req-0c79ade4-aa78-4524-9824-ff09c2a4860e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:56.444476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] The token cache is being invalidate because project 2a64e4583fa44c2c8a9ba72742fdd8ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:11:56.444774 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:56.445483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c79ade4-aa78-4524-9824-ff09c2a4860e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:56.447268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1228/2430] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:11:56 2022] DELETE /identity/v3/projects/2a64e4583fa44c2c8a9ba72742fdd8ac => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:11:56.452084 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd81666-d011-4c8e-90a5-b46bd78a1864 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:56.452547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd81666-d011-4c8e-90a5-b46bd78a1864 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:56.452642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd81666-d011-4c8e-90a5-b46bd78a1864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:56.476674 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0fd81666-d011-4c8e-90a5-b46bd78a1864 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:56.478087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0fd81666-d011-4c8e-90a5-b46bd78a1864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:56.546380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1203/2431] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:56.552438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24870ca5-6d9f-4796-bd96-3ef7f7178057 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:56.552735 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24870ca5-6d9f-4796-bd96-3ef7f7178057 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:56.553063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24870ca5-6d9f-4796-bd96-3ef7f7178057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:56.574925 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-24870ca5-6d9f-4796-bd96-3ef7f7178057 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:56.575273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-24870ca5-6d9f-4796-bd96-3ef7f7178057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:56.601948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1229/2432] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:56.602853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd81666-d011-4c8e-90a5-b46bd78a1864 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:56.614985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd81666-d011-4c8e-90a5-b46bd78a1864 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:56.620237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573bcc6-06c1-42a4-88e6-e30e57316504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:56.620951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573bcc6-06c1-42a4-88e6-e30e57316504 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:56.621263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573bcc6-06c1-42a4-88e6-e30e57316504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:56.621543 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573bcc6-06c1-42a4-88e6-e30e57316504 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:56.622379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573bcc6-06c1-42a4-88e6-e30e57316504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:11:56.625774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573bcc6-06c1-42a4-88e6-e30e57316504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:56.627769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573bcc6-06c1-42a4-88e6-e30e57316504 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:56.650520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24870ca5-6d9f-4796-bd96-3ef7f7178057 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:56.656868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1204/2433] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:11:56.661763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24870ca5-6d9f-4796-bd96-3ef7f7178057 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:56.666967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151b169-24be-431e-9f1e-4e4794b431a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:56.667591 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151b169-24be-431e-9f1e-4e4794b431a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:56.667873 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151b169-24be-431e-9f1e-4e4794b431a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:56.668149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151b169-24be-431e-9f1e-4e4794b431a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:56.668893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151b169-24be-431e-9f1e-4e4794b431a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:56.670996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151b169-24be-431e-9f1e-4e4794b431a2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:56.785595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1230/2434] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:11:56 2022] POST /identity/v3/projects => generated 361 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:56.789890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573bcc6-06c1-42a4-88e6-e30e57316504 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:56.801805 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573bcc6-06c1-42a4-88e6-e30e57316504 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:56.873259 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e5d92-da03-484e-ab2d-193e678d3a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:56.873798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5d92-da03-484e-ab2d-193e678d3a12 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:56.874077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5d92-da03-484e-ab2d-193e678d3a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:56.874353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5d92-da03-484e-ab2d-193e678d3a12 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:56.875042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e5d92-da03-484e-ab2d-193e678d3a12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:56.876994 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e5d92-da03-484e-ab2d-193e678d3a12 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:56.973366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1205/2435] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:11:56 2022] POST /identity/v3/users => generated 281 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:56.977101 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151b169-24be-431e-9f1e-4e4794b431a2 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:56.989930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151b169-24be-431e-9f1e-4e4794b431a2 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:56.995174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773889c-3a8e-48fb-989a-3a1b4e8c5220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:56.996010 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773889c-3a8e-48fb-989a-3a1b4e8c5220 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:56.996298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773889c-3a8e-48fb-989a-3a1b4e8c5220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:56.996568 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773889c-3a8e-48fb-989a-3a1b4e8c5220 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:56.997268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773889c-3a8e-48fb-989a-3a1b4e8c5220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:56.999071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773889c-3a8e-48fb-989a-3a1b4e8c5220 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:57.013813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1231/2436] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:11:57.015060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e5d92-da03-484e-ab2d-193e678d3a12 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:57.026895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e5d92-da03-484e-ab2d-193e678d3a12 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:57.033695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0931ed-dc69-48c3-ad9f-4d3e440f7402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:57.033695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0931ed-dc69-48c3-ad9f-4d3e440f7402 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:57.033695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0931ed-dc69-48c3-ad9f-4d3e440f7402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:57.033695 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0931ed-dc69-48c3-ad9f-4d3e440f7402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:57.034161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0931ed-dc69-48c3-ad9f-4d3e440f7402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:57.035682 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0931ed-dc69-48c3-ad9f-4d3e440f7402 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:57.050705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1206/2437] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:11:57.051958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773889c-3a8e-48fb-989a-3a1b4e8c5220 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:57.063877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773889c-3a8e-48fb-989a-3a1b4e8c5220 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:57.069850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8ebc7-8d95-4dd9-b773-29c9cffc0545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:57.070612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8ebc7-8d95-4dd9-b773-29c9cffc0545 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:57.070983 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8ebc7-8d95-4dd9-b773-29c9cffc0545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:57.071323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8ebc7-8d95-4dd9-b773-29c9cffc0545 admin admin] PATH_INFO: `/v3/projects/c5d35713ae6f4db1a2874336ee56588c/users/6b8eacda12d74da8ad4893686b15e026/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:57.093046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8ebc7-8d95-4dd9-b773-29c9cffc0545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d35713ae6f4db1a2874336ee56588c, user_id=6b8eacda12d74da8ad4893686b15e026, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:57.097752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8ebc7-8d95-4dd9-b773-29c9cffc0545 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:57.188134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1232/2438] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:11:57 2022] PUT /identity/v3/projects/c5d35713ae6f4db1a2874336ee56588c/users/6b8eacda12d74da8ad4893686b15e026/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:57.198978 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b970e5f-edcd-40d5-ba00-2e6b5aa302b8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:57.199457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b970e5f-edcd-40d5-ba00-2e6b5aa302b8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:57.199819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b970e5f-edcd-40d5-ba00-2e6b5aa302b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:57.232445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4b970e5f-edcd-40d5-ba00-2e6b5aa302b8 None None] MFA Rules not processed for user `6b8eacda12d74da8ad4893686b15e026`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:57.232445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4b970e5f-edcd-40d5-ba00-2e6b5aa302b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:57.238539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:57.359481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1207/2439] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:57.407971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8ebc7-8d95-4dd9-b773-29c9cffc0545 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:57.409929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c0a8ebc7-8d95-4dd9-b773-29c9cffc0545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:57.474331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8ebc7-8d95-4dd9-b773-29c9cffc0545 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:57.478379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579b759-2a21-43ed-9be8-2f83664b7484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:57.479026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579b759-2a21-43ed-9be8-2f83664b7484 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:57.479305 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579b759-2a21-43ed-9be8-2f83664b7484 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:57.479576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579b759-2a21-43ed-9be8-2f83664b7484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:57.490159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579b759-2a21-43ed-9be8-2f83664b7484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:57.491862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579b759-2a21-43ed-9be8-2f83664b7484 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:57.501906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1233/2440] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:57.518784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b970e5f-edcd-40d5-ba00-2e6b5aa302b8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:57.585948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b970e5f-edcd-40d5-ba00-2e6b5aa302b8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:57.591420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:57.592062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:57.592340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:57.592611 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] PATH_INFO: `/v3/users/6b8eacda12d74da8ad4893686b15e026` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:57.593801 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b8eacda12d74da8ad4893686b15e026)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:57.595806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:57.772684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b8eacda12d74da8ad4893686b15e026'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:57.786057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b8eacda12d74da8ad4893686b15e026'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:57.794098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b8eacda12d74da8ad4893686b15e026'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:57.897078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b8eacda12d74da8ad4893686b15e026'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:57.897820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b8eacda12d74da8ad4893686b15e026'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:57.907208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1208/2441] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:11:57 2022] DELETE /identity/v3/users/6b8eacda12d74da8ad4893686b15e026 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:57.910394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0a24-f1e9-4690-a6eb-ecf10638b943 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:57.910731 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0a24-f1e9-4690-a6eb-ecf10638b943 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:57.911013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0a24-f1e9-4690-a6eb-ecf10638b943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:57.919060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:57.930901 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8e9c0a24-f1e9-4690-a6eb-ecf10638b943 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:57.931839 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8e9c0a24-f1e9-4690-a6eb-ecf10638b943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:57.992106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1234/2442] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:58.001938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:58.003463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:58.035878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:58.065937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326776-1f2d-430a-91eb-cd4478cf3c2f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:58.069698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b6b40-743f-4db0-90ef-b16a757750b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:58.070288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b6b40-743f-4db0-90ef-b16a757750b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:58.070619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b6b40-743f-4db0-90ef-b16a757750b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:58.070913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b6b40-743f-4db0-90ef-b16a757750b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:58.079928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b6b40-743f-4db0-90ef-b16a757750b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:58.081854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b6b40-743f-4db0-90ef-b16a757750b1 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:58.091020 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1209/2443] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:58.360426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c0a24-f1e9-4690-a6eb-ecf10638b943 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:58.361931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8e9c0a24-f1e9-4690-a6eb-ecf10638b943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:58.437957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c0a24-f1e9-4690-a6eb-ecf10638b943 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:58.442655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafada1c-9e0a-441b-9498-2e3b7aad92b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:58.443295 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafada1c-9e0a-441b-9498-2e3b7aad92b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:58.443578 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafada1c-9e0a-441b-9498-2e3b7aad92b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:58.443856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafada1c-9e0a-441b-9498-2e3b7aad92b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:58.445586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cafada1c-9e0a-441b-9498-2e3b7aad92b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:58.506793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafada1c-9e0a-441b-9498-2e3b7aad92b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:58.508233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafada1c-9e0a-441b-9498-2e3b7aad92b1 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:58.517543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1235/2444] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:11:58.686704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b6b40-743f-4db0-90ef-b16a757750b1 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:58.688200 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5e8b6b40-743f-4db0-90ef-b16a757750b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:58.701487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b6b40-743f-4db0-90ef-b16a757750b1 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:58.707407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:58.708107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:58.708453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:58.708790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] PATH_INFO: `/v3/projects/c5d35713ae6f4db1a2874336ee56588c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:58.709977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5d35713ae6f4db1a2874336ee56588c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:58.712082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:58.845069 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5d35713ae6f4db1a2874336ee56588c', 'request_id': 'req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:58.851341 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5d35713ae6f4db1a2874336ee56588c', 'request_id': 'req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:58.856922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5d35713ae6f4db1a2874336ee56588c', 'request_id': 'req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:58.947897 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] The token cache is being invalidate because project c5d35713ae6f4db1a2874336ee56588c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:11:58.948192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:58.949326 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-90dd474e-d4b6-4d1d-9131-3b61adf31ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:11:58.950920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1210/2445] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:11:58 2022] DELETE /identity/v3/projects/c5d35713ae6f4db1a2874336ee56588c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:58.956271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc703e2-64c0-42a7-8972-da936232e6da None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:58.956684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc703e2-64c0-42a7-8972-da936232e6da None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:58.957035 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc703e2-64c0-42a7-8972-da936232e6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:58.977019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2fc703e2-64c0-42a7-8972-da936232e6da None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:58.978250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2fc703e2-64c0-42a7-8972-da936232e6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:59.030559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:59.038851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1236/2446] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:59.046641 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae19cf-1abd-4ca3-a1b3-67c10e6e063c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:59.047023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae19cf-1abd-4ca3-a1b3-67c10e6e063c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:59.047300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae19cf-1abd-4ca3-a1b3-67c10e6e063c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:59.069505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d7ae19cf-1abd-4ca3-a1b3-67c10e6e063c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:59.070824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d7ae19cf-1abd-4ca3-a1b3-67c10e6e063c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:59.078806 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1211/2447] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:59.083045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc703e2-64c0-42a7-8972-da936232e6da None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:59.092627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc703e2-64c0-42a7-8972-da936232e6da None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:59.097809 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28493a52-8c0b-4336-ad99-d2835d76fea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:59.098353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28493a52-8c0b-4336-ad99-d2835d76fea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:59.098635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28493a52-8c0b-4336-ad99-d2835d76fea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:59.098949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28493a52-8c0b-4336-ad99-d2835d76fea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:59.099620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28493a52-8c0b-4336-ad99-d2835d76fea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:11:59.099931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28493a52-8c0b-4336-ad99-d2835d76fea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:59.101545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28493a52-8c0b-4336-ad99-d2835d76fea3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:59.113589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1237/2448] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:11:59.117689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae19cf-1abd-4ca3-a1b3-67c10e6e063c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:59.127530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae19cf-1abd-4ca3-a1b3-67c10e6e063c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:59.132213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd316c9c-1605-46de-a5f2-98faf5c331e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:59.132825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd316c9c-1605-46de-a5f2-98faf5c331e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:59.133102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd316c9c-1605-46de-a5f2-98faf5c331e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:59.133391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd316c9c-1605-46de-a5f2-98faf5c331e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:59.134015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd316c9c-1605-46de-a5f2-98faf5c331e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:59.138432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd316c9c-1605-46de-a5f2-98faf5c331e7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:59.230486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1212/2449] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:11:59 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:59.234490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28493a52-8c0b-4336-ad99-d2835d76fea3 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:59.245309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28493a52-8c0b-4336-ad99-d2835d76fea3 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:59.303525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae64eb-e83c-4621-872b-683a3f77a967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:59.304117 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae64eb-e83c-4621-872b-683a3f77a967 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:59.304401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae64eb-e83c-4621-872b-683a3f77a967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:59.304742 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae64eb-e83c-4621-872b-683a3f77a967 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:59.305360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae64eb-e83c-4621-872b-683a3f77a967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:59.307311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae64eb-e83c-4621-872b-683a3f77a967 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:59.403477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1238/2450] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:11:59 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:11:59.407860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd316c9c-1605-46de-a5f2-98faf5c331e7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:59.418549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd316c9c-1605-46de-a5f2-98faf5c331e7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:59.423258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f84ec1-becc-4fe3-bb33-756dc1a69d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:59.425012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f84ec1-becc-4fe3-bb33-756dc1a69d67 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:59.425012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f84ec1-becc-4fe3-bb33-756dc1a69d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:59.425012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f84ec1-becc-4fe3-bb33-756dc1a69d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:59.425625 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f84ec1-becc-4fe3-bb33-756dc1a69d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:59.427234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f84ec1-becc-4fe3-bb33-756dc1a69d67 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:59.438072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1213/2451] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:11:59.442166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae64eb-e83c-4621-872b-683a3f77a967 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:59.452438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae64eb-e83c-4621-872b-683a3f77a967 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:59.457561 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c82bf-a99d-4804-8a07-a4e7f4b96997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:59.458273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c82bf-a99d-4804-8a07-a4e7f4b96997 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:59.461791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c82bf-a99d-4804-8a07-a4e7f4b96997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:59.461791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c82bf-a99d-4804-8a07-a4e7f4b96997 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:59.461791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c82bf-a99d-4804-8a07-a4e7f4b96997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:59.461791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c82bf-a99d-4804-8a07-a4e7f4b96997 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:59.472552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1239/2452] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:11:59.476769 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f84ec1-becc-4fe3-bb33-756dc1a69d67 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:59.487094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f84ec1-becc-4fe3-bb33-756dc1a69d67 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:59.491799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7eeac-d74b-451f-8cf8-2459248dfc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:59.492444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7eeac-d74b-451f-8cf8-2459248dfc6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:59.492726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7eeac-d74b-451f-8cf8-2459248dfc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:59.493002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7eeac-d74b-451f-8cf8-2459248dfc6d admin admin] PATH_INFO: `/v3/projects/dd7f67a691404532aea4fa3fc2d6c741/users/08c48039de494f3a869d62259f7e3ecc/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:59.507647 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7eeac-d74b-451f-8cf8-2459248dfc6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7f67a691404532aea4fa3fc2d6c741, user_id=08c48039de494f3a869d62259f7e3ecc, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:59.510291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7eeac-d74b-451f-8cf8-2459248dfc6d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:59.605486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1214/2453] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:11:59 2022] PUT /identity/v3/projects/dd7f67a691404532aea4fa3fc2d6c741/users/08c48039de494f3a869d62259f7e3ecc/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:11:59.616893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366462c-9a4f-4f6e-9dc1-f0c43ac38f19 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:59.617184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366462c-9a4f-4f6e-9dc1-f0c43ac38f19 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:59.617464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366462c-9a4f-4f6e-9dc1-f0c43ac38f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:59.645814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0366462c-9a4f-4f6e-9dc1-f0c43ac38f19 None None] MFA Rules not processed for user `08c48039de494f3a869d62259f7e3ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:11:59.647092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0366462c-9a4f-4f6e-9dc1-f0c43ac38f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:59.711791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1240/2454] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:11:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:11:59.724791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7eeac-d74b-451f-8cf8-2459248dfc6d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:11:59.727047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0aa7eeac-d74b-451f-8cf8-2459248dfc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:11:59.785928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:11:59.790931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7eeac-d74b-451f-8cf8-2459248dfc6d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:11:59.794768 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73606b-5afb-4c25-9487-7fdcad815e5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:11:59.795374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73606b-5afb-4c25-9487-7fdcad815e5c service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:11:59.795654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73606b-5afb-4c25-9487-7fdcad815e5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:11:59.795927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73606b-5afb-4c25-9487-7fdcad815e5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:11:59.804550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73606b-5afb-4c25-9487-7fdcad815e5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:11:59.806015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73606b-5afb-4c25-9487-7fdcad815e5c service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:11:59.815252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1215/2455] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:11:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:12:02.876061 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366462c-9a4f-4f6e-9dc1-f0c43ac38f19 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:02.878046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0366462c-9a4f-4f6e-9dc1-f0c43ac38f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:02.933756 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366462c-9a4f-4f6e-9dc1-f0c43ac38f19 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:02.937302 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5592cf3-2a34-4708-a86d-8d6d28f29be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:02.937906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5592cf3-2a34-4708-a86d-8d6d28f29be8 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:02.938186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5592cf3-2a34-4708-a86d-8d6d28f29be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:02.938459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5592cf3-2a34-4708-a86d-8d6d28f29be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:02.940652 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e5592cf3-2a34-4708-a86d-8d6d28f29be8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:02.949581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:03.004760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5592cf3-2a34-4708-a86d-8d6d28f29be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:03.006153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5592cf3-2a34-4708-a86d-8d6d28f29be8 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:03.015975 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1241/2456] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:03.759496 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73606b-5afb-4c25-9487-7fdcad815e5c service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:03.772370 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73606b-5afb-4c25-9487-7fdcad815e5c service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:03.777647 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:03.778418 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:03.779307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:03.779715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] PATH_INFO: `/v3/users/08c48039de494f3a869d62259f7e3ecc` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:03.780950 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c48039de494f3a869d62259f7e3ecc)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:03.783181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:03.999727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c48039de494f3a869d62259f7e3ecc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:04.014548 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c48039de494f3a869d62259f7e3ecc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:04.024982 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c48039de494f3a869d62259f7e3ecc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:04.108186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c48039de494f3a869d62259f7e3ecc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:04.108995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c48039de494f3a869d62259f7e3ecc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:04.116291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1216/2457] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:12:03 2022] DELETE /identity/v3/users/08c48039de494f3a869d62259f7e3ecc => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:04.120906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9993b61-b3ab-4587-8c43-efa346941fda None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:04.121386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9993b61-b3ab-4587-8c43-efa346941fda None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:04.121584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9993b61-b3ab-4587-8c43-efa346941fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:04.140216 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a9993b61-b3ab-4587-8c43-efa346941fda None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:04.141404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a9993b61-b3ab-4587-8c43-efa346941fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:04.202439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1242/2458] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:04.214034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:04.214221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:04.274138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b1ae5-c683-401a-bf6d-bf31fc124b27 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:04.277737 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a6d46-b265-4b76-ada0-9508a6f22be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:04.278344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a6d46-b265-4b76-ada0-9508a6f22be8 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:04.278819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a6d46-b265-4b76-ada0-9508a6f22be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:04.279319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a6d46-b265-4b76-ada0-9508a6f22be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:04.289251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a6d46-b265-4b76-ada0-9508a6f22be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:04.290706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a6d46-b265-4b76-ada0-9508a6f22be8 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:04.300624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1217/2459] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:04.730054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9993b61-b3ab-4587-8c43-efa346941fda None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:04.732220 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a9993b61-b3ab-4587-8c43-efa346941fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:04.750243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9993b61-b3ab-4587-8c43-efa346941fda None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:04.755756 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:04.756847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:04.757217 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:04.757599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] PATH_INFO: `/v3/projects/dd7f67a691404532aea4fa3fc2d6c741` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:04.758926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd7f67a691404532aea4fa3fc2d6c741)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:04.761081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:04.884247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd7f67a691404532aea4fa3fc2d6c741', 'request_id': 'req-c74ea3b9-e758-4e64-b7ad-039e645f4b22'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:04.892804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd7f67a691404532aea4fa3fc2d6c741', 'request_id': 'req-c74ea3b9-e758-4e64-b7ad-039e645f4b22'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:04.900580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd7f67a691404532aea4fa3fc2d6c741', 'request_id': 'req-c74ea3b9-e758-4e64-b7ad-039e645f4b22'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:04.975943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] The token cache is being invalidate because project dd7f67a691404532aea4fa3fc2d6c741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:12:04.976382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:04.977053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c74ea3b9-e758-4e64-b7ad-039e645f4b22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:04.978606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1243/2460] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:12:04 2022] DELETE /identity/v3/projects/dd7f67a691404532aea4fa3fc2d6c741 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:04.983835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce98eb4-5b9c-414d-98c1-7d485623b88a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:04.984379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce98eb4-5b9c-414d-98c1-7d485623b88a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:04.984379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce98eb4-5b9c-414d-98c1-7d485623b88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:05.009112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7ce98eb4-5b9c-414d-98c1-7d485623b88a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:05.010875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7ce98eb4-5b9c-414d-98c1-7d485623b88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:05.092283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1218/2461] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:05.100435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2f3c-ad73-431e-8579-774bd59ab308 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:05.100828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2f3c-ad73-431e-8579-774bd59ab308 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:05.101028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2f3c-ad73-431e-8579-774bd59ab308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:05.128500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-109a2f3c-ad73-431e-8579-774bd59ab308 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:05.131391 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-109a2f3c-ad73-431e-8579-774bd59ab308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:05.156631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1244/2462] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:05.157264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce98eb4-5b9c-414d-98c1-7d485623b88a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:05.164506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:05.168700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce98eb4-5b9c-414d-98c1-7d485623b88a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:05.174568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae824580-c8e3-4d12-a663-0468da81c725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:05.175214 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae824580-c8e3-4d12-a663-0468da81c725 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:05.175497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae824580-c8e3-4d12-a663-0468da81c725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:05.176789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae824580-c8e3-4d12-a663-0468da81c725 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:05.176789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae824580-c8e3-4d12-a663-0468da81c725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:12:05.176789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae824580-c8e3-4d12-a663-0468da81c725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:05.178309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae824580-c8e3-4d12-a663-0468da81c725 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:05.193744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1219/2463] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:12:05.195414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a2f3c-ad73-431e-8579-774bd59ab308 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:05.205780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a2f3c-ad73-431e-8579-774bd59ab308 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:05.211133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a6b03-40d4-45a9-8308-176069b3449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:05.213208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a6b03-40d4-45a9-8308-176069b3449a admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:05.213766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a6b03-40d4-45a9-8308-176069b3449a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:05.215378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a6b03-40d4-45a9-8308-176069b3449a admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:05.216946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a6b03-40d4-45a9-8308-176069b3449a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:05.219577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a6b03-40d4-45a9-8308-176069b3449a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:05.319696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:05.326110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1245/2464] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:12:05 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:05.330474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae824580-c8e3-4d12-a663-0468da81c725 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:05.340401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae824580-c8e3-4d12-a663-0468da81c725 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:05.398592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc323ae1-a001-44e9-ace3-6ed2c647468d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:05.399247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc323ae1-a001-44e9-ace3-6ed2c647468d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:05.399530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc323ae1-a001-44e9-ace3-6ed2c647468d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:05.399805 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc323ae1-a001-44e9-ace3-6ed2c647468d admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:05.400459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc323ae1-a001-44e9-ace3-6ed2c647468d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:05.402427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc323ae1-a001-44e9-ace3-6ed2c647468d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:05.514536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1220/2465] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:12:05 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:05.518753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a6b03-40d4-45a9-8308-176069b3449a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:05.529593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a6b03-40d4-45a9-8308-176069b3449a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:05.535146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a63e3-06fd-43fd-9815-f849af9f4858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:05.538900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a63e3-06fd-43fd-9815-f849af9f4858 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:05.538900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a63e3-06fd-43fd-9815-f849af9f4858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:05.538900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a63e3-06fd-43fd-9815-f849af9f4858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:05.538900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a63e3-06fd-43fd-9815-f849af9f4858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:05.538900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a63e3-06fd-43fd-9815-f849af9f4858 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:05.549038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1246/2466] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:05.553080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc323ae1-a001-44e9-ace3-6ed2c647468d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:05.565537 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc323ae1-a001-44e9-ace3-6ed2c647468d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:05.570802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15bda2-e7fa-4033-b4e6-2b7795976f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:05.571833 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15bda2-e7fa-4033-b4e6-2b7795976f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:05.572311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15bda2-e7fa-4033-b4e6-2b7795976f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:05.572773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15bda2-e7fa-4033-b4e6-2b7795976f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:05.573614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15bda2-e7fa-4033-b4e6-2b7795976f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:05.575615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15bda2-e7fa-4033-b4e6-2b7795976f9a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:05.596709 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1221/2467] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:05.598873 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a63e3-06fd-43fd-9815-f849af9f4858 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:05.609598 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a63e3-06fd-43fd-9815-f849af9f4858 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:05.614598 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcc3bb-4745-47e0-8322-4177cef966cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:05.615314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcc3bb-4745-47e0-8322-4177cef966cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:05.615596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcc3bb-4745-47e0-8322-4177cef966cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:05.615881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcc3bb-4745-47e0-8322-4177cef966cb admin admin] PATH_INFO: `/v3/projects/9e69c6ce72224091825c73731d0211b9/users/a9feaba0fd1149ec8779fc36fb49bf64/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:05.632500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcc3bb-4745-47e0-8322-4177cef966cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e69c6ce72224091825c73731d0211b9, user_id=a9feaba0fd1149ec8779fc36fb49bf64, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:05.635647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcc3bb-4745-47e0-8322-4177cef966cb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:05.708618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1247/2468] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:05 2022] PUT /identity/v3/projects/9e69c6ce72224091825c73731d0211b9/users/a9feaba0fd1149ec8779fc36fb49bf64/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:12:05.719192 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bc805-d256-4301-81b3-f508c4c10cd7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:05.719523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bc805-d256-4301-81b3-f508c4c10cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:05.719766 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bc805-d256-4301-81b3-f508c4c10cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:05.749700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5a9bc805-d256-4301-81b3-f508c4c10cd7 None None] MFA Rules not processed for user `a9feaba0fd1149ec8779fc36fb49bf64`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:05.751008 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5a9bc805-d256-4301-81b3-f508c4c10cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:05.817132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1222/2469] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:05.829234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcc3bb-4745-47e0-8322-4177cef966cb admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:05.831848 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-61bcc3bb-4745-47e0-8322-4177cef966cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:05.890177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcc3bb-4745-47e0-8322-4177cef966cb admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:05.894450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca1dff-f152-496d-964e-d6f16c5eed53 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c214b2c97a7b41698b433eac8379442a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:05.895086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca1dff-f152-496d-964e-d6f16c5eed53 service glance] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:05.895364 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca1dff-f152-496d-964e-d6f16c5eed53 service glance] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:05.895635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca1dff-f152-496d-964e-d6f16c5eed53 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:05.903744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca1dff-f152-496d-964e-d6f16c5eed53 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:05.905413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca1dff-f152-496d-964e-d6f16c5eed53 service glance] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:05.914194 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1248/2470] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:06.595596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bc805-d256-4301-81b3-f508c4c10cd7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:06.597721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5a9bc805-d256-4301-81b3-f508c4c10cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:06.664067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:06.669824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bc805-d256-4301-81b3-f508c4c10cd7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:06.675421 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fa1e6-0bb9-48d8-921d-b62585c18eec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:06.676028 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fa1e6-0bb9-48d8-921d-b62585c18eec service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:06.676312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fa1e6-0bb9-48d8-921d-b62585c18eec service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:06.676581 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fa1e6-0bb9-48d8-921d-b62585c18eec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:06.678394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ef9fa1e6-0bb9-48d8-921d-b62585c18eec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:06.689663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fa1e6-0bb9-48d8-921d-b62585c18eec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:06.691303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fa1e6-0bb9-48d8-921d-b62585c18eec service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:06.701060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1223/2471] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:06.715884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca1dff-f152-496d-964e-d6f16c5eed53 service glance] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:06.717468 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-49ca1dff-f152-496d-964e-d6f16c5eed53 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:06.810855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca1dff-f152-496d-964e-d6f16c5eed53 service glance] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:06.815276 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1420b5-02a2-4c5e-b436-afaf38df47d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:06.815900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1420b5-02a2-4c5e-b436-afaf38df47d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:06.816177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1420b5-02a2-4c5e-b436-afaf38df47d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:06.816444 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1420b5-02a2-4c5e-b436-afaf38df47d2 admin admin] PATH_INFO: `/v3/users/393fddf9c75447fab334b84694dcf4f1` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:06.817567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1420b5-02a2-4c5e-b436-afaf38df47d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=393fddf9c75447fab334b84694dcf4f1)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:06.819488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1420b5-02a2-4c5e-b436-afaf38df47d2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:07.024935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9d1420b5-02a2-4c5e-b436-afaf38df47d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '393fddf9c75447fab334b84694dcf4f1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:07.037593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9d1420b5-02a2-4c5e-b436-afaf38df47d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '393fddf9c75447fab334b84694dcf4f1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:07.042050 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:07.045121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9d1420b5-02a2-4c5e-b436-afaf38df47d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '393fddf9c75447fab334b84694dcf4f1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:07.137056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9d1420b5-02a2-4c5e-b436-afaf38df47d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '393fddf9c75447fab334b84694dcf4f1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:07.137884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9d1420b5-02a2-4c5e-b436-afaf38df47d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '393fddf9c75447fab334b84694dcf4f1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:07.144880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1249/2472] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:12:06 2022] DELETE /identity/v3/users/393fddf9c75447fab334b84694dcf4f1 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:07.496302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fa1e6-0bb9-48d8-921d-b62585c18eec service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:07.511262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fa1e6-0bb9-48d8-921d-b62585c18eec service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:07.595106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:07.595911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:07.596256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:07.596815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] PATH_INFO: `/v3/projects/45838a0f741e485e8f424375c6d0d489` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:07.597879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45838a0f741e485e8f424375c6d0d489)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:07.600147 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:07.732257 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45838a0f741e485e8f424375c6d0d489', 'request_id': 'req-838b1915-0cd7-44bf-856f-08c7dbade857'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:07.739112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45838a0f741e485e8f424375c6d0d489', 'request_id': 'req-838b1915-0cd7-44bf-856f-08c7dbade857'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:07.744905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45838a0f741e485e8f424375c6d0d489', 'request_id': 'req-838b1915-0cd7-44bf-856f-08c7dbade857'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:07.827422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] The token cache is being invalidate because project 45838a0f741e485e8f424375c6d0d489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:12:07.827723 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:07.828460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-838b1915-0cd7-44bf-856f-08c7dbade857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:07.830089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1224/2473] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:12:07 2022] DELETE /identity/v3/projects/45838a0f741e485e8f424375c6d0d489 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:07.835323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668a151-9cc8-4273-a7c1-78a4c7adf5be None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:07.835786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668a151-9cc8-4273-a7c1-78a4c7adf5be None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:07.836128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668a151-9cc8-4273-a7c1-78a4c7adf5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:07.857982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f668a151-9cc8-4273-a7c1-78a4c7adf5be None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:07.859294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f668a151-9cc8-4273-a7c1-78a4c7adf5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:07.920924 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1250/2474] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:07.930315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2078c1-2d7b-450b-93fd-facba08caec8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:07.930569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2078c1-2d7b-450b-93fd-facba08caec8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:07.930829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2078c1-2d7b-450b-93fd-facba08caec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:07.952350 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0a2078c1-2d7b-450b-93fd-facba08caec8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:07.953826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0a2078c1-2d7b-450b-93fd-facba08caec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:07.962517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1225/2475] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:07.966933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668a151-9cc8-4273-a7c1-78a4c7adf5be None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:07.976812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668a151-9cc8-4273-a7c1-78a4c7adf5be None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:07.981887 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ebce3-f978-431c-a5b7-beb84e8dfe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:07.982543 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ebce3-f978-431c-a5b7-beb84e8dfe7e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:07.982860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ebce3-f978-431c-a5b7-beb84e8dfe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:07.983137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ebce3-f978-431c-a5b7-beb84e8dfe7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:07.984036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ebce3-f978-431c-a5b7-beb84e8dfe7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:12:07.984441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ebce3-f978-431c-a5b7-beb84e8dfe7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:07.986063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ebce3-f978-431c-a5b7-beb84e8dfe7e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:08.002730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1251/2476] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:12:08.004009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2078c1-2d7b-450b-93fd-facba08caec8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:08.013526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2078c1-2d7b-450b-93fd-facba08caec8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:08.018463 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4fe79-f6ff-4c40-8e3e-4ee6272f42eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:08.020765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4fe79-f6ff-4c40-8e3e-4ee6272f42eb admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:08.020765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4fe79-f6ff-4c40-8e3e-4ee6272f42eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:08.020765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4fe79-f6ff-4c40-8e3e-4ee6272f42eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:08.020765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4fe79-f6ff-4c40-8e3e-4ee6272f42eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:08.022248 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4fe79-f6ff-4c40-8e3e-4ee6272f42eb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:08.098340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1226/2477] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:12:08 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:08.104006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ebce3-f978-431c-a5b7-beb84e8dfe7e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:08.115157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ebce3-f978-431c-a5b7-beb84e8dfe7e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:08.172372 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea9848-98f7-42e2-934c-68359ae04526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:08.172985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea9848-98f7-42e2-934c-68359ae04526 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:08.173294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea9848-98f7-42e2-934c-68359ae04526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:08.173565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea9848-98f7-42e2-934c-68359ae04526 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:08.175230 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea9848-98f7-42e2-934c-68359ae04526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:08.177143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea9848-98f7-42e2-934c-68359ae04526 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:08.303596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1252/2478] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:12:08 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:08.307658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4fe79-f6ff-4c40-8e3e-4ee6272f42eb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:08.318157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4fe79-f6ff-4c40-8e3e-4ee6272f42eb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:08.322667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22adf1-d589-4e1d-83b6-445996ddeda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:08.323287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22adf1-d589-4e1d-83b6-445996ddeda6 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:08.323564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22adf1-d589-4e1d-83b6-445996ddeda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:08.323836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22adf1-d589-4e1d-83b6-445996ddeda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:08.324474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22adf1-d589-4e1d-83b6-445996ddeda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:08.326073 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22adf1-d589-4e1d-83b6-445996ddeda6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:08.336655 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1227/2479] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:08.340616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea9848-98f7-42e2-934c-68359ae04526 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:08.351362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea9848-98f7-42e2-934c-68359ae04526 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:08.356029 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398adfe-791e-45ad-b29b-920696871380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:08.356643 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398adfe-791e-45ad-b29b-920696871380 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:08.358876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398adfe-791e-45ad-b29b-920696871380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:08.358876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398adfe-791e-45ad-b29b-920696871380 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:08.358876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398adfe-791e-45ad-b29b-920696871380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:08.359724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398adfe-791e-45ad-b29b-920696871380 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:08.371890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1253/2480] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:08.376235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22adf1-d589-4e1d-83b6-445996ddeda6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:08.386480 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22adf1-d589-4e1d-83b6-445996ddeda6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:08.391332 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdab72-708e-4811-b1ef-065b007b5350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:08.391989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdab72-708e-4811-b1ef-065b007b5350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:08.392313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdab72-708e-4811-b1ef-065b007b5350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:08.392591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdab72-708e-4811-b1ef-065b007b5350 admin admin] PATH_INFO: `/v3/projects/a6e1d253ff8041ecbb96a3cba86b17a6/users/291d4784a1ac4762b3247f277b1f7ec0/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:08.408011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdab72-708e-4811-b1ef-065b007b5350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e1d253ff8041ecbb96a3cba86b17a6, user_id=291d4784a1ac4762b3247f277b1f7ec0, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:08.411291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdab72-708e-4811-b1ef-065b007b5350 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:08.479385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1228/2481] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:08 2022] PUT /identity/v3/projects/a6e1d253ff8041ecbb96a3cba86b17a6/users/291d4784a1ac4762b3247f277b1f7ec0/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:12:08.489671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7f6fc-ff42-4a2f-8e57-7e14c1f24d8f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:08.489945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7f6fc-ff42-4a2f-8e57-7e14c1f24d8f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:08.490217 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7f6fc-ff42-4a2f-8e57-7e14c1f24d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:08.518997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-23d7f6fc-ff42-4a2f-8e57-7e14c1f24d8f None None] MFA Rules not processed for user `291d4784a1ac4762b3247f277b1f7ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:08.520331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-23d7f6fc-ff42-4a2f-8e57-7e14c1f24d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:08.564120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:08.585921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1254/2482] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:08.598097 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdab72-708e-4811-b1ef-065b007b5350 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:08.599962 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7bcdab72-708e-4811-b1ef-065b007b5350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:08.620279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:08.682547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdab72-708e-4811-b1ef-065b007b5350 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:08.690226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a50fd-661a-4415-8341-122f689f4100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:08.690899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a50fd-661a-4415-8341-122f689f4100 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:08.691183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a50fd-661a-4415-8341-122f689f4100 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:08.691450 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a50fd-661a-4415-8341-122f689f4100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:08.700362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a50fd-661a-4415-8341-122f689f4100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:08.702144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a50fd-661a-4415-8341-122f689f4100 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:08.713161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1229/2483] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:11.823549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7f6fc-ff42-4a2f-8e57-7e14c1f24d8f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:11.892576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7f6fc-ff42-4a2f-8e57-7e14c1f24d8f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:11.897647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:11.898280 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:11.898702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:11.899073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] PATH_INFO: `/v3/users/291d4784a1ac4762b3247f277b1f7ec0` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:11.900244 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=291d4784a1ac4762b3247f277b1f7ec0)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:11.902199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:12.079483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291d4784a1ac4762b3247f277b1f7ec0'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:12.091724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291d4784a1ac4762b3247f277b1f7ec0'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:12.099225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291d4784a1ac4762b3247f277b1f7ec0'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:12.147785 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291d4784a1ac4762b3247f277b1f7ec0'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:12.148438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291d4784a1ac4762b3247f277b1f7ec0'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:12.155229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1255/2484] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:12:11 2022] DELETE /identity/v3/users/291d4784a1ac4762b3247f277b1f7ec0 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:12.160776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc248e-46b4-4820-b1af-af300b5ddc8c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:12.161273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc248e-46b4-4820-b1af-af300b5ddc8c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:12.161371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc248e-46b4-4820-b1af-af300b5ddc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:12.181075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f7dc248e-46b4-4820-b1af-af300b5ddc8c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:12.182324 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f7dc248e-46b4-4820-b1af-af300b5ddc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:12.249220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1230/2485] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:12.258897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:12.260337 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:12.329273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1a8fc-4c69-4c76-b448-8260a2107617 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:12.333944 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5eb68-ff35-49f2-abdf-f9c2d490e2ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:12.335320 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5eb68-ff35-49f2-abdf-f9c2d490e2ef service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:12.335608 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5eb68-ff35-49f2-abdf-f9c2d490e2ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:12.335884 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5eb68-ff35-49f2-abdf-f9c2d490e2ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:12.348282 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5eb68-ff35-49f2-abdf-f9c2d490e2ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:12.349673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5eb68-ff35-49f2-abdf-f9c2d490e2ef service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:12.361158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1256/2486] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:12.773028 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc248e-46b4-4820-b1af-af300b5ddc8c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:12.774488 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f7dc248e-46b4-4820-b1af-af300b5ddc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:12.790402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc248e-46b4-4820-b1af-af300b5ddc8c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:12.795101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:12.795831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:12.796110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:12.796382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] PATH_INFO: `/v3/projects/a6e1d253ff8041ecbb96a3cba86b17a6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:12.797896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e1d253ff8041ecbb96a3cba86b17a6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:12.801382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:12.909141 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e1d253ff8041ecbb96a3cba86b17a6', 'request_id': 'req-4b56d6db-6949-4141-bd15-a4f60ee99e7b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:12.915419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e1d253ff8041ecbb96a3cba86b17a6', 'request_id': 'req-4b56d6db-6949-4141-bd15-a4f60ee99e7b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:12.920847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e1d253ff8041ecbb96a3cba86b17a6', 'request_id': 'req-4b56d6db-6949-4141-bd15-a4f60ee99e7b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:12.985904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] The token cache is being invalidate because project a6e1d253ff8041ecbb96a3cba86b17a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:12:12.986193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:12.986834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b56d6db-6949-4141-bd15-a4f60ee99e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:12.988831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1231/2487] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:12:12 2022] DELETE /identity/v3/projects/a6e1d253ff8041ecbb96a3cba86b17a6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:12.994004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30fbd-36ee-449d-82d6-435f0dcbe3e7 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:12.994511 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30fbd-36ee-449d-82d6-435f0dcbe3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:12.994611 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30fbd-36ee-449d-82d6-435f0dcbe3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:13.004673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:13.015971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e5d30fbd-36ee-449d-82d6-435f0dcbe3e7 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:13.017180 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e5d30fbd-36ee-449d-82d6-435f0dcbe3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:13.079676 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1257/2488] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:13.087199 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8938d-3108-40d9-b862-2238ee5328df None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:13.087516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8938d-3108-40d9-b862-2238ee5328df None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:13.087797 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8938d-3108-40d9-b862-2238ee5328df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:13.109322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f2b8938d-3108-40d9-b862-2238ee5328df None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:13.110555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f2b8938d-3108-40d9-b862-2238ee5328df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:13.119553 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1232/2489] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:13.124216 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d30fbd-36ee-449d-82d6-435f0dcbe3e7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:13.135096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d30fbd-36ee-449d-82d6-435f0dcbe3e7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:13.139732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b59132-f688-4ca1-bd5d-a9937ce8671d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:13.148054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b59132-f688-4ca1-bd5d-a9937ce8671d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:13.148054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b59132-f688-4ca1-bd5d-a9937ce8671d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:13.148054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b59132-f688-4ca1-bd5d-a9937ce8671d admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:13.148054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b59132-f688-4ca1-bd5d-a9937ce8671d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:12:13.148054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b59132-f688-4ca1-bd5d-a9937ce8671d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:13.148054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b59132-f688-4ca1-bd5d-a9937ce8671d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:13.157212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1258/2490] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:12:13.162205 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8938d-3108-40d9-b862-2238ee5328df None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:13.171940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8938d-3108-40d9-b862-2238ee5328df None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:13.178203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d82f2-f0ca-4ff9-b383-0313608d3bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:13.178854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d82f2-f0ca-4ff9-b383-0313608d3bdc admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:13.179129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d82f2-f0ca-4ff9-b383-0313608d3bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:13.179397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d82f2-f0ca-4ff9-b383-0313608d3bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:13.180047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d82f2-f0ca-4ff9-b383-0313608d3bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:13.182030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d82f2-f0ca-4ff9-b383-0313608d3bdc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:13.198237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:13.316217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1233/2491] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:12:13 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:13.320462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b59132-f688-4ca1-bd5d-a9937ce8671d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:13.331545 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b59132-f688-4ca1-bd5d-a9937ce8671d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:13.391297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219310f-053a-4861-b580-feddae7f7470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:13.392477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219310f-053a-4861-b580-feddae7f7470 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:13.392477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219310f-053a-4861-b580-feddae7f7470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:13.392477 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219310f-053a-4861-b580-feddae7f7470 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:13.393178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219310f-053a-4861-b580-feddae7f7470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:13.395660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219310f-053a-4861-b580-feddae7f7470 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:13.475409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1259/2492] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:12:13 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:13.476780 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d82f2-f0ca-4ff9-b383-0313608d3bdc admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:13.488012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d82f2-f0ca-4ff9-b383-0313608d3bdc admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:13.493232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce14c1d-8ede-4120-a8ff-aae975e8a7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:13.493951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce14c1d-8ede-4120-a8ff-aae975e8a7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:13.494391 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce14c1d-8ede-4120-a8ff-aae975e8a7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:13.494810 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce14c1d-8ede-4120-a8ff-aae975e8a7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:13.495575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce14c1d-8ede-4120-a8ff-aae975e8a7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:13.497444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce14c1d-8ede-4120-a8ff-aae975e8a7b5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:13.514043 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1234/2493] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:13.515678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219310f-053a-4861-b580-feddae7f7470 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:13.527191 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219310f-053a-4861-b580-feddae7f7470 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:13.534953 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2e926-c3bf-42f9-ad1a-8176beb9cb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:13.535602 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2e926-c3bf-42f9-ad1a-8176beb9cb66 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:13.536003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2e926-c3bf-42f9-ad1a-8176beb9cb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:13.536385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2e926-c3bf-42f9-ad1a-8176beb9cb66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:13.537287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2e926-c3bf-42f9-ad1a-8176beb9cb66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:13.539040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2e926-c3bf-42f9-ad1a-8176beb9cb66 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:13.552828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1260/2494] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:13.557796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce14c1d-8ede-4120-a8ff-aae975e8a7b5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:13.572546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce14c1d-8ede-4120-a8ff-aae975e8a7b5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:13.578126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc0810-87c6-4d24-a883-314b3836705a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:13.578830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc0810-87c6-4d24-a883-314b3836705a admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:13.579095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc0810-87c6-4d24-a883-314b3836705a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:13.579375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc0810-87c6-4d24-a883-314b3836705a admin admin] PATH_INFO: `/v3/projects/b4dd748a933e4c24a85624347d2586fd/users/c8ca0777c691427e8249dbb3a69e17a3/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:13.602056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc0810-87c6-4d24-a883-314b3836705a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4dd748a933e4c24a85624347d2586fd, user_id=c8ca0777c691427e8249dbb3a69e17a3, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:13.604675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc0810-87c6-4d24-a883-314b3836705a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:13.691782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1235/2495] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:13 2022] PUT /identity/v3/projects/b4dd748a933e4c24a85624347d2586fd/users/c8ca0777c691427e8249dbb3a69e17a3/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:12:13.703207 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290f285-3f94-4588-82a5-c67434f11099 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:13.703496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290f285-3f94-4588-82a5-c67434f11099 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:13.703774 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290f285-3f94-4588-82a5-c67434f11099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:13.731931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a290f285-3f94-4588-82a5-c67434f11099 None None] MFA Rules not processed for user `c8ca0777c691427e8249dbb3a69e17a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:13.733172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a290f285-3f94-4588-82a5-c67434f11099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:13.801700 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:13.805497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1261/2496] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:13.816902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc0810-87c6-4d24-a883-314b3836705a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:13.818646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dcbc0810-87c6-4d24-a883-314b3836705a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:13.905948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc0810-87c6-4d24-a883-314b3836705a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:13.911698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe456b86-f82b-4f9d-a262-3b8876502cea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:13.912311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe456b86-f82b-4f9d-a262-3b8876502cea service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:13.912735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe456b86-f82b-4f9d-a262-3b8876502cea service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:13.913021 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe456b86-f82b-4f9d-a262-3b8876502cea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:13.921045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe456b86-f82b-4f9d-a262-3b8876502cea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:13.923087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe456b86-f82b-4f9d-a262-3b8876502cea service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:13.932836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1236/2497] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:14.141162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290f285-3f94-4588-82a5-c67434f11099 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:14.211187 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290f285-3f94-4588-82a5-c67434f11099 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:14.215937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:14.216639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:14.216924 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:14.217198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] PATH_INFO: `/v3/users/c8ca0777c691427e8249dbb3a69e17a3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:14.218343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ca0777c691427e8249dbb3a69e17a3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:14.220355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:14.414567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ca0777c691427e8249dbb3a69e17a3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:14.427058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ca0777c691427e8249dbb3a69e17a3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:14.433869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ca0777c691427e8249dbb3a69e17a3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:14.536402 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ca0777c691427e8249dbb3a69e17a3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:14.537027 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ca0777c691427e8249dbb3a69e17a3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:14.543667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1262/2498] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:12:14 2022] DELETE /identity/v3/users/c8ca0777c691427e8249dbb3a69e17a3 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:14.548176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7674-6230-41d1-961f-0a10dea2ea2e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:14.548653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7674-6230-41d1-961f-0a10dea2ea2e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:14.548749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7674-6230-41d1-961f-0a10dea2ea2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:14.567957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d24f7674-6230-41d1-961f-0a10dea2ea2e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:14.569227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d24f7674-6230-41d1-961f-0a10dea2ea2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:14.640875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1237/2499] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:14.651092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:14.655998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:14.714432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8ccae-983b-495e-a4e3-c79131ba9fbd admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:14.718144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d67d67-32d2-4076-81be-4509ff2477da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:14.718779 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d67d67-32d2-4076-81be-4509ff2477da service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:14.719065 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d67d67-32d2-4076-81be-4509ff2477da service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:14.719339 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d67d67-32d2-4076-81be-4509ff2477da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:14.733973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d67d67-32d2-4076-81be-4509ff2477da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:14.735610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d67d67-32d2-4076-81be-4509ff2477da service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:14.746663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1263/2500] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:15.124467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f7674-6230-41d1-961f-0a10dea2ea2e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:15.126230 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d24f7674-6230-41d1-961f-0a10dea2ea2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:15.139320 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f7674-6230-41d1-961f-0a10dea2ea2e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:15.144253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:15.145058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:15.145444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:15.145601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] PATH_INFO: `/v3/projects/b4dd748a933e4c24a85624347d2586fd` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:15.146788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4dd748a933e4c24a85624347d2586fd)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:15.148704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:15.261557 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:15.267568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4dd748a933e4c24a85624347d2586fd', 'request_id': 'req-af018779-ac8b-4d1a-9075-5831819b80be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:15.276280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4dd748a933e4c24a85624347d2586fd', 'request_id': 'req-af018779-ac8b-4d1a-9075-5831819b80be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:15.282142 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4dd748a933e4c24a85624347d2586fd', 'request_id': 'req-af018779-ac8b-4d1a-9075-5831819b80be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:15.398722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] The token cache is being invalidate because project b4dd748a933e4c24a85624347d2586fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:12:15.399019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:15.399745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af018779-ac8b-4d1a-9075-5831819b80be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:15.401242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1238/2501] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:12:15 2022] DELETE /identity/v3/projects/b4dd748a933e4c24a85624347d2586fd => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:12:15.406479 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9546d-976b-4955-9997-0cba6d358699 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:15.407000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9546d-976b-4955-9997-0cba6d358699 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:15.407223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9546d-976b-4955-9997-0cba6d358699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:15.428275 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-79b9546d-976b-4955-9997-0cba6d358699 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:15.429493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-79b9546d-976b-4955-9997-0cba6d358699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:15.446753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:15.491822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1264/2502] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:15.499893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115dde9-494b-4fbf-a01f-c71ea03185ef None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:15.500277 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115dde9-494b-4fbf-a01f-c71ea03185ef None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:15.500573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115dde9-494b-4fbf-a01f-c71ea03185ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:15.521188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2115dde9-494b-4fbf-a01f-c71ea03185ef None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:15.522551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2115dde9-494b-4fbf-a01f-c71ea03185ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:15.531158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1239/2503] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:12:15.536110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9546d-976b-4955-9997-0cba6d358699 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:15.546391 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9546d-976b-4955-9997-0cba6d358699 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:15.550977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965565d-0a79-419b-bc89-93b0a7b60c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:15.551659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965565d-0a79-419b-bc89-93b0a7b60c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:15.551943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965565d-0a79-419b-bc89-93b0a7b60c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:15.552217 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965565d-0a79-419b-bc89-93b0a7b60c71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:15.553058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965565d-0a79-419b-bc89-93b0a7b60c71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:12:15.553368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965565d-0a79-419b-bc89-93b0a7b60c71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:15.555254 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965565d-0a79-419b-bc89-93b0a7b60c71 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:15.568199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1265/2504] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:12:15.572735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115dde9-494b-4fbf-a01f-c71ea03185ef None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:15.582738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115dde9-494b-4fbf-a01f-c71ea03185ef None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:15.587454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87768788-f60a-486f-8419-d47a54c9aca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:15.588063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87768788-f60a-486f-8419-d47a54c9aca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:15.588338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87768788-f60a-486f-8419-d47a54c9aca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:15.588607 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87768788-f60a-486f-8419-d47a54c9aca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:15.589248 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87768788-f60a-486f-8419-d47a54c9aca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:15.591187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87768788-f60a-486f-8419-d47a54c9aca2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:15.675748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1240/2505] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:12:15 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:15.680104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965565d-0a79-419b-bc89-93b0a7b60c71 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:15.690947 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965565d-0a79-419b-bc89-93b0a7b60c71 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:15.749901 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b537a-554c-430d-b397-46bfd2b50d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:15.750464 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b537a-554c-430d-b397-46bfd2b50d33 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:15.750776 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b537a-554c-430d-b397-46bfd2b50d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:15.751054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b537a-554c-430d-b397-46bfd2b50d33 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:15.751765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b537a-554c-430d-b397-46bfd2b50d33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:15.754441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b537a-554c-430d-b397-46bfd2b50d33 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:15.837258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1266/2506] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:12:15 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:15.838642 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87768788-f60a-486f-8419-d47a54c9aca2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:15.850160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87768788-f60a-486f-8419-d47a54c9aca2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:15.854819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87007d52-e3e1-4a2f-b034-ba5405be23ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:15.855423 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87007d52-e3e1-4a2f-b034-ba5405be23ee admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:15.855702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87007d52-e3e1-4a2f-b034-ba5405be23ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:15.855978 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87007d52-e3e1-4a2f-b034-ba5405be23ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:15.856616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87007d52-e3e1-4a2f-b034-ba5405be23ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:15.858229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87007d52-e3e1-4a2f-b034-ba5405be23ee admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:15.869790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1241/2507] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:15.873931 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b537a-554c-430d-b397-46bfd2b50d33 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:15.884422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b537a-554c-430d-b397-46bfd2b50d33 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:15.889100 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d980f41-357f-43de-8690-64f24eced64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:15.889701 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d980f41-357f-43de-8690-64f24eced64a admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:15.889987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d980f41-357f-43de-8690-64f24eced64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:15.890259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d980f41-357f-43de-8690-64f24eced64a admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:15.891104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d980f41-357f-43de-8690-64f24eced64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:15.892832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d980f41-357f-43de-8690-64f24eced64a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:15.907212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1267/2508] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:15.911879 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87007d52-e3e1-4a2f-b034-ba5405be23ee admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:15.922388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87007d52-e3e1-4a2f-b034-ba5405be23ee admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:15.930280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb3b0b-762d-4e66-879c-21f5304dc0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:15.930994 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb3b0b-762d-4e66-879c-21f5304dc0ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:15.931276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb3b0b-762d-4e66-879c-21f5304dc0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:15.931559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb3b0b-762d-4e66-879c-21f5304dc0ab admin admin] PATH_INFO: `/v3/projects/a3d2df082d444cef906ca8de14226e2b/users/8cc1bc584bc54bf1abd5e859c1e33ad5/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:15.946506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb3b0b-762d-4e66-879c-21f5304dc0ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d2df082d444cef906ca8de14226e2b, user_id=8cc1bc584bc54bf1abd5e859c1e33ad5, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:15.949315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb3b0b-762d-4e66-879c-21f5304dc0ab admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:16.011369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1242/2509] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:15 2022] PUT /identity/v3/projects/a3d2df082d444cef906ca8de14226e2b/users/8cc1bc584bc54bf1abd5e859c1e33ad5/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:16.022469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1aeb7c-0bf8-495c-a987-8cbc35defa22 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:16.022791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1aeb7c-0bf8-495c-a987-8cbc35defa22 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:16.023070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1aeb7c-0bf8-495c-a987-8cbc35defa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:16.051727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-bd1aeb7c-0bf8-495c-a987-8cbc35defa22 None None] MFA Rules not processed for user `8cc1bc584bc54bf1abd5e859c1e33ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:16.052999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bd1aeb7c-0bf8-495c-a987-8cbc35defa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:16.119104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1268/2510] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:16.123587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb3b0b-762d-4e66-879c-21f5304dc0ab admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:16.185571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb3b0b-762d-4e66-879c-21f5304dc0ab admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:16.191029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacda1ad-ad67-48c3-9c42-04fa7e2debd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:16.191615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacda1ad-ad67-48c3-9c42-04fa7e2debd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:16.191896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacda1ad-ad67-48c3-9c42-04fa7e2debd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:16.192310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacda1ad-ad67-48c3-9c42-04fa7e2debd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:16.192955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacda1ad-ad67-48c3-9c42-04fa7e2debd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:16.195373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacda1ad-ad67-48c3-9c42-04fa7e2debd6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:16.275364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1243/2511] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:12:16 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:16.279865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1aeb7c-0bf8-495c-a987-8cbc35defa22 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:16.291322 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1aeb7c-0bf8-495c-a987-8cbc35defa22 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:16.304595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:16.350158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596bf0c-bb0b-492c-8c0d-0ca7b20b6de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:16.350801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596bf0c-bb0b-492c-8c0d-0ca7b20b6de0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:16.351083 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596bf0c-bb0b-492c-8c0d-0ca7b20b6de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:16.351358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596bf0c-bb0b-492c-8c0d-0ca7b20b6de0 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:16.352009 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596bf0c-bb0b-492c-8c0d-0ca7b20b6de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:16.353951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596bf0c-bb0b-492c-8c0d-0ca7b20b6de0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:16.434111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1269/2512] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:12:16 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:12:16.438148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacda1ad-ad67-48c3-9c42-04fa7e2debd6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:16.448662 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacda1ad-ad67-48c3-9c42-04fa7e2debd6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:16.453431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff38a0-d12f-4fa6-ab53-c3a7d07bf060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:16.454025 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff38a0-d12f-4fa6-ab53-c3a7d07bf060 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:16.454533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff38a0-d12f-4fa6-ab53-c3a7d07bf060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:16.454867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff38a0-d12f-4fa6-ab53-c3a7d07bf060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:16.455635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff38a0-d12f-4fa6-ab53-c3a7d07bf060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:16.457419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff38a0-d12f-4fa6-ab53-c3a7d07bf060 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:16.468667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1244/2513] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:16.474253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596bf0c-bb0b-492c-8c0d-0ca7b20b6de0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:16.488596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596bf0c-bb0b-492c-8c0d-0ca7b20b6de0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:16.493255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0e08f-3a3b-4a53-a57a-b3c7ec09fe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:16.493921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0e08f-3a3b-4a53-a57a-b3c7ec09fe11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:16.494226 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0e08f-3a3b-4a53-a57a-b3c7ec09fe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:16.494497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0e08f-3a3b-4a53-a57a-b3c7ec09fe11 admin admin] PATH_INFO: `/v3/projects/e8375ffe835046b8a7ca3a658923c0da/users/894274013a2a4aea99435a67f9528ab0/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:16.510515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0e08f-3a3b-4a53-a57a-b3c7ec09fe11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8375ffe835046b8a7ca3a658923c0da, user_id=894274013a2a4aea99435a67f9528ab0, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:16.513194 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0e08f-3a3b-4a53-a57a-b3c7ec09fe11 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:16.582893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1270/2514] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:16 2022] PUT /identity/v3/projects/e8375ffe835046b8a7ca3a658923c0da/users/894274013a2a4aea99435a67f9528ab0/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:12:16.587085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff38a0-d12f-4fa6-ab53-c3a7d07bf060 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:16.605691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff38a0-d12f-4fa6-ab53-c3a7d07bf060 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:16.645921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:16.664292 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a723d5-6288-428f-9413-a5fa7218d5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:16.664907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a723d5-6288-428f-9413-a5fa7218d5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:16.665190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a723d5-6288-428f-9413-a5fa7218d5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:16.665466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a723d5-6288-428f-9413-a5fa7218d5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:16.666108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a723d5-6288-428f-9413-a5fa7218d5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:16.667784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a723d5-6288-428f-9413-a5fa7218d5f2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:16.678935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1245/2515] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:16.682157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0e08f-3a3b-4a53-a57a-b3c7ec09fe11 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:16.693340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0e08f-3a3b-4a53-a57a-b3c7ec09fe11 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:16.697913 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002e901-1ebe-4625-98d4-c2c07bd569f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:16.698560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002e901-1ebe-4625-98d4-c2c07bd569f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:16.698877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002e901-1ebe-4625-98d4-c2c07bd569f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:16.699177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002e901-1ebe-4625-98d4-c2c07bd569f1 admin admin] PATH_INFO: `/v3/projects/e8375ffe835046b8a7ca3a658923c0da/users/894274013a2a4aea99435a67f9528ab0/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:16.701177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002e901-1ebe-4625-98d4-c2c07bd569f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8375ffe835046b8a7ca3a658923c0da, user_id=894274013a2a4aea99435a67f9528ab0, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:16.704038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002e901-1ebe-4625-98d4-c2c07bd569f1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:16.714828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1271/2516] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:16 2022] PUT /identity/v3/projects/e8375ffe835046b8a7ca3a658923c0da/users/894274013a2a4aea99435a67f9528ab0/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:16.726652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66941903-7bf4-4949-9ead-e1d7b60a7f98 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:16.727001 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66941903-7bf4-4949-9ead-e1d7b60a7f98 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:16.727284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66941903-7bf4-4949-9ead-e1d7b60a7f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:16.759742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-66941903-7bf4-4949-9ead-e1d7b60a7f98 None None] MFA Rules not processed for user `894274013a2a4aea99435a67f9528ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:16.761075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-66941903-7bf4-4949-9ead-e1d7b60a7f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:16.833131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1246/2517] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:16.843681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002e901-1ebe-4625-98d4-c2c07bd569f1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:16.845530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1002e901-1ebe-4625-98d4-c2c07bd569f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:16.905474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002e901-1ebe-4625-98d4-c2c07bd569f1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:16.909119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f0cfa-6924-4467-a5f0-ca0d5ebf82b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:16.909725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f0cfa-6924-4467-a5f0-ca0d5ebf82b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:16.910001 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f0cfa-6924-4467-a5f0-ca0d5ebf82b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:16.910836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f0cfa-6924-4467-a5f0-ca0d5ebf82b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:16.966770 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f0cfa-6924-4467-a5f0-ca0d5ebf82b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:16.968203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f0cfa-6924-4467-a5f0-ca0d5ebf82b1 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:16.977589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1272/2518] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:20.087691 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66941903-7bf4-4949-9ead-e1d7b60a7f98 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:20.147891 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66941903-7bf4-4949-9ead-e1d7b60a7f98 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:20.152443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:20.153060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:20.153337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:20.153604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] PATH_INFO: `/v3/users/8cc1bc584bc54bf1abd5e859c1e33ad5` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:20.154981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc1bc584bc54bf1abd5e859c1e33ad5)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:20.156930 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:20.326221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc1bc584bc54bf1abd5e859c1e33ad5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:20.337197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:20.338536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc1bc584bc54bf1abd5e859c1e33ad5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:20.345639 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc1bc584bc54bf1abd5e859c1e33ad5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:20.398388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc1bc584bc54bf1abd5e859c1e33ad5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:20.399002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc1bc584bc54bf1abd5e859c1e33ad5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:20.405135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1247/2519] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:12:20 2022] DELETE /identity/v3/users/8cc1bc584bc54bf1abd5e859c1e33ad5 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:20.409743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ebb83-4d0e-4a0f-920c-d19a194cfefc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:20.410235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ebb83-4d0e-4a0f-920c-d19a194cfefc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:20.410343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ebb83-4d0e-4a0f-920c-d19a194cfefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:20.429474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-680ebb83-4d0e-4a0f-920c-d19a194cfefc None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:20.430712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-680ebb83-4d0e-4a0f-920c-d19a194cfefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:20.448498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:20.488707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1273/2520] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:20.498100 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:20.499565 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:20.556618 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1dc2e-b10d-4a44-baeb-7b1162e36f13 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:20.560204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d6b2-b167-40a5-8764-0e07b1dce77f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:20.560799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d6b2-b167-40a5-8764-0e07b1dce77f service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:20.561076 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d6b2-b167-40a5-8764-0e07b1dce77f service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:20.561345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d6b2-b167-40a5-8764-0e07b1dce77f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:20.570219 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78d6b2-b167-40a5-8764-0e07b1dce77f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:20.571669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78d6b2-b167-40a5-8764-0e07b1dce77f service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:20.580944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1248/2521] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:21.007147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ebb83-4d0e-4a0f-920c-d19a194cfefc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:21.008665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-680ebb83-4d0e-4a0f-920c-d19a194cfefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:21.020914 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ebb83-4d0e-4a0f-920c-d19a194cfefc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:21.026193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:21.026870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:21.027154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:21.027475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] PATH_INFO: `/v3/projects/a3d2df082d444cef906ca8de14226e2b` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:21.028603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d2df082d444cef906ca8de14226e2b)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:21.030811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:21.154658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d2df082d444cef906ca8de14226e2b', 'request_id': 'req-ee3b59f6-ad23-4775-b203-2ac0a0084f48'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:21.161017 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d2df082d444cef906ca8de14226e2b', 'request_id': 'req-ee3b59f6-ad23-4775-b203-2ac0a0084f48'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:21.166643 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d2df082d444cef906ca8de14226e2b', 'request_id': 'req-ee3b59f6-ad23-4775-b203-2ac0a0084f48'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:21.244269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] The token cache is being invalidate because project a3d2df082d444cef906ca8de14226e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:12:21.244558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:21.245230 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:21.246750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1274/2522] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:12:21 2022] DELETE /identity/v3/projects/a3d2df082d444cef906ca8de14226e2b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:12:21.250791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d6b2-b167-40a5-8764-0e07b1dce77f service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:21.262154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d6b2-b167-40a5-8764-0e07b1dce77f service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:21.318846 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf74af-199e-445a-bd6c-70993b89fa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:21.319515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf74af-199e-445a-bd6c-70993b89fa6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:21.319791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf74af-199e-445a-bd6c-70993b89fa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:21.320083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf74af-199e-445a-bd6c-70993b89fa6b admin admin] PATH_INFO: `/v3/users/894274013a2a4aea99435a67f9528ab0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:21.321206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf74af-199e-445a-bd6c-70993b89fa6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=894274013a2a4aea99435a67f9528ab0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:21.323172 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf74af-199e-445a-bd6c-70993b89fa6b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:21.519999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4dcf74af-199e-445a-bd6c-70993b89fa6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '894274013a2a4aea99435a67f9528ab0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:21.532187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4dcf74af-199e-445a-bd6c-70993b89fa6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '894274013a2a4aea99435a67f9528ab0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:21.539903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4dcf74af-199e-445a-bd6c-70993b89fa6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '894274013a2a4aea99435a67f9528ab0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:21.610273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4dcf74af-199e-445a-bd6c-70993b89fa6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '894274013a2a4aea99435a67f9528ab0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:21.610911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4dcf74af-199e-445a-bd6c-70993b89fa6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '894274013a2a4aea99435a67f9528ab0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:21.617462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1249/2523] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:12:21 2022] DELETE /identity/v3/users/894274013a2a4aea99435a67f9528ab0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:22.058043 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:22.059504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:22.074664 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b59f6-ad23-4775-b203-2ac0a0084f48 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:22.134513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:22.135236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:22.135520 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:22.135817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] PATH_INFO: `/v3/projects/e8375ffe835046b8a7ca3a658923c0da` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:22.137263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8375ffe835046b8a7ca3a658923c0da)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:22.139214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:22.241055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8375ffe835046b8a7ca3a658923c0da', 'request_id': 'req-35dfec90-c3de-49e4-a18f-1860a1cd1495'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:22.242193 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:22.248336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8375ffe835046b8a7ca3a658923c0da', 'request_id': 'req-35dfec90-c3de-49e4-a18f-1860a1cd1495'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:22.254028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8375ffe835046b8a7ca3a658923c0da', 'request_id': 'req-35dfec90-c3de-49e4-a18f-1860a1cd1495'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:22.314340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] The token cache is being invalidate because project e8375ffe835046b8a7ca3a658923c0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:12:22.314637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:22.315427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-35dfec90-c3de-49e4-a18f-1860a1cd1495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:22.316959 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1275/2524] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:12:22 2022] DELETE /identity/v3/projects/e8375ffe835046b8a7ca3a658923c0da => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:22.321908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607e010-4e42-4c84-bf90-92677d6f9878 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:22.322404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607e010-4e42-4c84-bf90-92677d6f9878 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:22.322509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607e010-4e42-4c84-bf90-92677d6f9878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:22.330992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:22.343437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e607e010-4e42-4c84-bf90-92677d6f9878 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:22.344646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e607e010-4e42-4c84-bf90-92677d6f9878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:22.405394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1250/2525] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:22.413627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771933c-da52-4be3-a2f9-69475be517f1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:22.413916 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771933c-da52-4be3-a2f9-69475be517f1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:22.414221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771933c-da52-4be3-a2f9-69475be517f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:22.434251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c771933c-da52-4be3-a2f9-69475be517f1 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:22.435506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c771933c-da52-4be3-a2f9-69475be517f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:22.443430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1276/2526] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:22.447528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607e010-4e42-4c84-bf90-92677d6f9878 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:22.457011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607e010-4e42-4c84-bf90-92677d6f9878 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:22.461854 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605e367-8c4a-4538-a3a7-772bac9ddaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:22.462455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605e367-8c4a-4538-a3a7-772bac9ddaf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:22.462772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605e367-8c4a-4538-a3a7-772bac9ddaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:22.467937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605e367-8c4a-4538-a3a7-772bac9ddaf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:22.467937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605e367-8c4a-4538-a3a7-772bac9ddaf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:12:22.467937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605e367-8c4a-4538-a3a7-772bac9ddaf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:22.467937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605e367-8c4a-4538-a3a7-772bac9ddaf5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:22.477644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1251/2527] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:12:22.481967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771933c-da52-4be3-a2f9-69475be517f1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:22.491900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771933c-da52-4be3-a2f9-69475be517f1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:22.496481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e7186-ace0-4627-b4ce-a28f390732a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:22.497071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e7186-ace0-4627-b4ce-a28f390732a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:22.497352 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e7186-ace0-4627-b4ce-a28f390732a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:22.497621 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e7186-ace0-4627-b4ce-a28f390732a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:22.498253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e7186-ace0-4627-b4ce-a28f390732a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:22.500182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e7186-ace0-4627-b4ce-a28f390732a3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:22.582730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1277/2528] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:12:22 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:22.586812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605e367-8c4a-4538-a3a7-772bac9ddaf5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:22.597352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605e367-8c4a-4538-a3a7-772bac9ddaf5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:22.654489 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1554d-5b0a-4dae-8d5e-2ec771560048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:22.655120 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1554d-5b0a-4dae-8d5e-2ec771560048 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:22.655398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1554d-5b0a-4dae-8d5e-2ec771560048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:22.655668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1554d-5b0a-4dae-8d5e-2ec771560048 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:22.656316 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1554d-5b0a-4dae-8d5e-2ec771560048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:22.658211 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1554d-5b0a-4dae-8d5e-2ec771560048 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:22.724674 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1252/2529] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:12:22 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:12:22.728733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e7186-ace0-4627-b4ce-a28f390732a3 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:22.739400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e7186-ace0-4627-b4ce-a28f390732a3 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:22.744125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5e97b-2316-465a-95f5-b2fbd9032c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:22.744724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5e97b-2316-465a-95f5-b2fbd9032c15 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:22.745005 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5e97b-2316-465a-95f5-b2fbd9032c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:22.745279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5e97b-2316-465a-95f5-b2fbd9032c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:22.745921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5e97b-2316-465a-95f5-b2fbd9032c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:22.747666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5e97b-2316-465a-95f5-b2fbd9032c15 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:22.757981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1278/2530] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:22.761913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1554d-5b0a-4dae-8d5e-2ec771560048 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:22.772746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1554d-5b0a-4dae-8d5e-2ec771560048 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:22.777443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764db19-e579-4ab9-9253-97a3f8832898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:22.778036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764db19-e579-4ab9-9253-97a3f8832898 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:22.778318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764db19-e579-4ab9-9253-97a3f8832898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:22.778888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764db19-e579-4ab9-9253-97a3f8832898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:22.779551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764db19-e579-4ab9-9253-97a3f8832898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:22.781223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764db19-e579-4ab9-9253-97a3f8832898 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:22.792774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1253/2531] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:22.796803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5e97b-2316-465a-95f5-b2fbd9032c15 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:22.806759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5e97b-2316-465a-95f5-b2fbd9032c15 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:22.811519 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838008c-417f-4f4d-9eaf-b511040162e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:22.812157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838008c-417f-4f4d-9eaf-b511040162e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:22.812439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838008c-417f-4f4d-9eaf-b511040162e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:22.812714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838008c-417f-4f4d-9eaf-b511040162e9 admin admin] PATH_INFO: `/v3/projects/9d32f40c2bd64a61aadd27f82d11191a/users/e9cd1b533aa8456d95e1bdbeac678b9f/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:22.827448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838008c-417f-4f4d-9eaf-b511040162e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d32f40c2bd64a61aadd27f82d11191a, user_id=e9cd1b533aa8456d95e1bdbeac678b9f, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:22.830281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838008c-417f-4f4d-9eaf-b511040162e9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:22.913813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1279/2532] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:22 2022] PUT /identity/v3/projects/9d32f40c2bd64a61aadd27f82d11191a/users/e9cd1b533aa8456d95e1bdbeac678b9f/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:22.918911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0a82c-68d8-4fa5-b31c-e2a33b4b7ab9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:22.919264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0a82c-68d8-4fa5-b31c-e2a33b4b7ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:22.919547 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0a82c-68d8-4fa5-b31c-e2a33b4b7ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:22.939652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d9d0a82c-68d8-4fa5-b31c-e2a33b4b7ab9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:22.940904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d9d0a82c-68d8-4fa5-b31c-e2a33b4b7ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:23.005063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1254/2533] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:12:23.017527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838008c-417f-4f4d-9eaf-b511040162e9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:23.017527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f838008c-417f-4f4d-9eaf-b511040162e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:23.081208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838008c-417f-4f4d-9eaf-b511040162e9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:23.086378 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5fd96-aea7-410f-946e-31e854ee13b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:23.086966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5fd96-aea7-410f-946e-31e854ee13b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:23.087255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5fd96-aea7-410f-946e-31e854ee13b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:23.087529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5fd96-aea7-410f-946e-31e854ee13b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:23.097996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5fd96-aea7-410f-946e-31e854ee13b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:23.099599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5fd96-aea7-410f-946e-31e854ee13b0 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:23.110725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1280/2534] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:12:24.554837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebff4b-bbb6-4eef-91da-d1270105d457 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:24.555983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebff4b-bbb6-4eef-91da-d1270105d457 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:24.555983 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebff4b-bbb6-4eef-91da-d1270105d457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:24.578451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:24.588143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-99ebff4b-bbb6-4eef-91da-d1270105d457 None None] MFA Rules not processed for user `e9cd1b533aa8456d95e1bdbeac678b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:24.589393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-99ebff4b-bbb6-4eef-91da-d1270105d457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:24.658404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1255/2535] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:24.665973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5fd96-aea7-410f-946e-31e854ee13b0 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:24.667380 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6ee5fd96-aea7-410f-946e-31e854ee13b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:24.727749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5fd96-aea7-410f-946e-31e854ee13b0 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:24.731503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af4b21-38f4-42db-bb08-16d3c51c7046 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:24.732109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af4b21-38f4-42db-bb08-16d3c51c7046 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:24.732387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af4b21-38f4-42db-bb08-16d3c51c7046 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:24.732656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af4b21-38f4-42db-bb08-16d3c51c7046 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:24.741358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af4b21-38f4-42db-bb08-16d3c51c7046 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:24.742827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af4b21-38f4-42db-bb08-16d3c51c7046 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:24.751618 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1281/2536] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:25.773264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d7528-097c-45f3-a53e-f8cd7951769f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:25.773719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d7528-097c-45f3-a53e-f8cd7951769f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:25.773827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d7528-097c-45f3-a53e-f8cd7951769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:25.803676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-135d7528-097c-45f3-a53e-f8cd7951769f None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:25.804907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-135d7528-097c-45f3-a53e-f8cd7951769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:25.876464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1256/2537] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:12:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:25.980988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af4b21-38f4-42db-bb08-16d3c51c7046 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:25.982455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-56af4b21-38f4-42db-bb08-16d3c51c7046 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:26.004135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:26.057812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af4b21-38f4-42db-bb08-16d3c51c7046 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:26.063207 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a20ab-1791-448c-8bd5-06a0b64515ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:26.063845 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a20ab-1791-448c-8bd5-06a0b64515ce service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:26.064129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a20ab-1791-448c-8bd5-06a0b64515ce service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:26.064409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a20ab-1791-448c-8bd5-06a0b64515ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:26.073788 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a20ab-1791-448c-8bd5-06a0b64515ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:26.075426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a20ab-1791-448c-8bd5-06a0b64515ce service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:26.085179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1282/2538] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:12:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 16 08:12:26.566909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2e88-0f63-4d75-92b4-6cc4f0dfc916 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:26.567539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2e88-0f63-4d75-92b4-6cc4f0dfc916 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:26.568306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2e88-0f63-4d75-92b4-6cc4f0dfc916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:26.590462 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3a3f2e88-0f63-4d75-92b4-6cc4f0dfc916 None None] MFA Rules not processed for user `e9cd1b533aa8456d95e1bdbeac678b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:26.592920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3a3f2e88-0f63-4d75-92b4-6cc4f0dfc916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:26.605859 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1257/2539] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:36.170225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=23 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:36.585171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=60 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:37.155891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a20ab-1791-448c-8bd5-06a0b64515ce service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:37.157549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d35a20ab-1791-448c-8bd5-06a0b64515ce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:37.172175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a20ab-1791-448c-8bd5-06a0b64515ce service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:37.177286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:37.178012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:37.178297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:37.178580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] PATH_INFO: `/v3/users/a9feaba0fd1149ec8779fc36fb49bf64` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:37.181094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9feaba0fd1149ec8779fc36fb49bf64)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:37.181768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:37.415395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9feaba0fd1149ec8779fc36fb49bf64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:37.434581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9feaba0fd1149ec8779fc36fb49bf64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:37.437929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9feaba0fd1149ec8779fc36fb49bf64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:37.513365 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9feaba0fd1149ec8779fc36fb49bf64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:37.514396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9feaba0fd1149ec8779fc36fb49bf64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:37.522599 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1283/2540] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:12:37 2022] DELETE /identity/v3/users/a9feaba0fd1149ec8779fc36fb49bf64 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:12:37.527160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095eb960-0239-4107-b55c-86c9d410c364 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:37.527676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095eb960-0239-4107-b55c-86c9d410c364 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:37.528140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095eb960-0239-4107-b55c-86c9d410c364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:37.573067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-095eb960-0239-4107-b55c-86c9d410c364 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:37.574444 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-095eb960-0239-4107-b55c-86c9d410c364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:37.642880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1258/2541] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:37.653113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:37.655463 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:37.731113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a993d-1506-439b-bba7-f0a7f7c3ea94 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:37.740439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b00e99-477c-41b0-a18f-36746cb08dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:37.742329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00e99-477c-41b0-a18f-36746cb08dca service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:37.742447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00e99-477c-41b0-a18f-36746cb08dca service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:37.742638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00e99-477c-41b0-a18f-36746cb08dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:37.753531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b00e99-477c-41b0-a18f-36746cb08dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:37.755019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b00e99-477c-41b0-a18f-36746cb08dca service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:37.776273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1284/2542] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:38.256035 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095eb960-0239-4107-b55c-86c9d410c364 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:38.258043 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-095eb960-0239-4107-b55c-86c9d410c364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:38.271532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095eb960-0239-4107-b55c-86c9d410c364 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:38.279231 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:38.280095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:38.280528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:38.280959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] PATH_INFO: `/v3/projects/9e69c6ce72224091825c73731d0211b9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:38.282412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e69c6ce72224091825c73731d0211b9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:38.284990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:38.434526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e69c6ce72224091825c73731d0211b9', 'request_id': 'req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:38.441087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e69c6ce72224091825c73731d0211b9', 'request_id': 'req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:38.447357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e69c6ce72224091825c73731d0211b9', 'request_id': 'req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:38.530173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] The token cache is being invalidate because project 9e69c6ce72224091825c73731d0211b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:12:38.530173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:38.542207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4059f3ff-7901-4afe-bee8-037f0ff2cf1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:38.542207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1259/2543] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:12:38 2022] DELETE /identity/v3/projects/9e69c6ce72224091825c73731d0211b9 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:12:38.542620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabfa66-71af-4a52-a0a1-d7243cef38dd None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:38.542620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabfa66-71af-4a52-a0a1-d7243cef38dd None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:38.542620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabfa66-71af-4a52-a0a1-d7243cef38dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:38.561168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ceabfa66-71af-4a52-a0a1-d7243cef38dd None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:38.562363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ceabfa66-71af-4a52-a0a1-d7243cef38dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:38.623362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1285/2544] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:38.631152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cedbb-3b64-420b-b037-dd3b563d317d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:38.632067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cedbb-3b64-420b-b037-dd3b563d317d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:38.632349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cedbb-3b64-420b-b037-dd3b563d317d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:38.652704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f09cedbb-3b64-420b-b037-dd3b563d317d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:38.654290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f09cedbb-3b64-420b-b037-dd3b563d317d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:38.662592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1260/2545] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:38.667028 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabfa66-71af-4a52-a0a1-d7243cef38dd None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:38.676909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabfa66-71af-4a52-a0a1-d7243cef38dd None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:38.682243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322f228-7da8-4137-a29e-21a45b64369c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:38.682881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f228-7da8-4137-a29e-21a45b64369c admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:38.683166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f228-7da8-4137-a29e-21a45b64369c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:38.691977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322f228-7da8-4137-a29e-21a45b64369c admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:38.691977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322f228-7da8-4137-a29e-21a45b64369c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:12:38.691977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322f228-7da8-4137-a29e-21a45b64369c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:38.691977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322f228-7da8-4137-a29e-21a45b64369c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:38.698624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1286/2546] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:12:38.703057 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cedbb-3b64-420b-b037-dd3b563d317d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:38.712451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cedbb-3b64-420b-b037-dd3b563d317d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:38.717455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b3e92-05cb-4980-8c14-1683bfdefebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:38.718063 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b3e92-05cb-4980-8c14-1683bfdefebb admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:38.718366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b3e92-05cb-4980-8c14-1683bfdefebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:38.718637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b3e92-05cb-4980-8c14-1683bfdefebb admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:38.719913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b3e92-05cb-4980-8c14-1683bfdefebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:38.721698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b3e92-05cb-4980-8c14-1683bfdefebb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:38.801015 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1261/2547] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:12:38 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:38.806111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322f228-7da8-4137-a29e-21a45b64369c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:38.811759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:38.817033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322f228-7da8-4137-a29e-21a45b64369c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:38.876829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33e13b-8477-4f4f-bafb-91884ab550bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:38.877447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33e13b-8477-4f4f-bafb-91884ab550bc admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:38.881529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33e13b-8477-4f4f-bafb-91884ab550bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:38.881529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33e13b-8477-4f4f-bafb-91884ab550bc admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:38.881529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33e13b-8477-4f4f-bafb-91884ab550bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:38.881529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33e13b-8477-4f4f-bafb-91884ab550bc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:38.934129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b3e92-05cb-4980-8c14-1683bfdefebb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:38.935871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e53b3e92-05cb-4980-8c14-1683bfdefebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:38.996372 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b3e92-05cb-4980-8c14-1683bfdefebb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:39.000518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a94d9a-aee3-4c85-8418-0235bbfd93da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:39.001122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94d9a-aee3-4c85-8418-0235bbfd93da service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:39.001405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94d9a-aee3-4c85-8418-0235bbfd93da service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:39.001552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1287/2548] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:12:38 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:39.001683 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a94d9a-aee3-4c85-8418-0235bbfd93da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:39.003545 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-69a94d9a-aee3-4c85-8418-0235bbfd93da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:39.005927 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33e13b-8477-4f4f-bafb-91884ab550bc admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:39.015891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33e13b-8477-4f4f-bafb-91884ab550bc admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:39.016712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a94d9a-aee3-4c85-8418-0235bbfd93da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:39.018240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a94d9a-aee3-4c85-8418-0235bbfd93da service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:39.020810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df692a3-a649-4a4c-bb78-241bd29066ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:39.021503 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df692a3-a649-4a4c-bb78-241bd29066ec admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:39.021869 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df692a3-a649-4a4c-bb78-241bd29066ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:39.022185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df692a3-a649-4a4c-bb78-241bd29066ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:39.026790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:39.026942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df692a3-a649-4a4c-bb78-241bd29066ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:39.026942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df692a3-a649-4a4c-bb78-241bd29066ec admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:39.029249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1262/2549] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:12:39.036627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1288/2550] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:12:39.041156 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a94d9a-aee3-4c85-8418-0235bbfd93da service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:39.051164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a94d9a-aee3-4c85-8418-0235bbfd93da service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:39.056620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4f421-cc09-498a-a414-caa59c175ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:39.057217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4f421-cc09-498a-a414-caa59c175ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:39.057528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4f421-cc09-498a-a414-caa59c175ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:39.057805 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4f421-cc09-498a-a414-caa59c175ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:39.058446 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4f421-cc09-498a-a414-caa59c175ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:39.060165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4f421-cc09-498a-a414-caa59c175ba0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:39.073401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1263/2551] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:39.074572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df692a3-a649-4a4c-bb78-241bd29066ec admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:39.085753 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df692a3-a649-4a4c-bb78-241bd29066ec admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:39.090704 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd25892-6889-4dd1-962a-d2b5899573ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:39.091449 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd25892-6889-4dd1-962a-d2b5899573ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:39.091805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd25892-6889-4dd1-962a-d2b5899573ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:39.092148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd25892-6889-4dd1-962a-d2b5899573ea admin admin] PATH_INFO: `/v3/projects/e3276981b48d4974841dd97ee1cb9882/users/cc14169c8f7044c6815944af5bc658a9/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:39.236287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd25892-6889-4dd1-962a-d2b5899573ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3276981b48d4974841dd97ee1cb9882, user_id=cc14169c8f7044c6815944af5bc658a9, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:39.239041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd25892-6889-4dd1-962a-d2b5899573ea admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:39.346167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1289/2552] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:39 2022] PUT /identity/v3/projects/e3276981b48d4974841dd97ee1cb9882/users/cc14169c8f7044c6815944af5bc658a9/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:39.352227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f861593-ac36-499d-928d-70ee8e56f74c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:39.352530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f861593-ac36-499d-928d-70ee8e56f74c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:39.352809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f861593-ac36-499d-928d-70ee8e56f74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:39.404242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0f861593-ac36-499d-928d-70ee8e56f74c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:39.406714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0f861593-ac36-499d-928d-70ee8e56f74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:39.515367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1264/2553] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:12:39.530408 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd25892-6889-4dd1-962a-d2b5899573ea admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:39.534071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5bd25892-6889-4dd1-962a-d2b5899573ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:39.611099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd25892-6889-4dd1-962a-d2b5899573ea admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:39.615702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974ee75-e9fd-43f2-ad37-1b65b91aa5db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:39.617303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974ee75-e9fd-43f2-ad37-1b65b91aa5db service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:39.617798 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974ee75-e9fd-43f2-ad37-1b65b91aa5db service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:39.618281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974ee75-e9fd-43f2-ad37-1b65b91aa5db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:39.628709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974ee75-e9fd-43f2-ad37-1b65b91aa5db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:39.630504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974ee75-e9fd-43f2-ad37-1b65b91aa5db service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:39.643418 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1290/2554] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:41.416672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d78f0-8b92-4913-a08f-16d937d5fa65 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:41.417539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d78f0-8b92-4913-a08f-16d937d5fa65 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:41.418013 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d78f0-8b92-4913-a08f-16d937d5fa65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:41.451309 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-298d78f0-8b92-4913-a08f-16d937d5fa65 None None] MFA Rules not processed for user `cc14169c8f7044c6815944af5bc658a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:41.452894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-298d78f0-8b92-4913-a08f-16d937d5fa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:41.571176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1265/2555] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:41.583504 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974ee75-e9fd-43f2-ad37-1b65b91aa5db service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:41.586149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f974ee75-e9fd-43f2-ad37-1b65b91aa5db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:41.648489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974ee75-e9fd-43f2-ad37-1b65b91aa5db service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:41.652122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d68a1-0c83-4da0-8f83-a7aeeab391e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:41.652724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d68a1-0c83-4da0-8f83-a7aeeab391e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:41.653006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d68a1-0c83-4da0-8f83-a7aeeab391e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:41.653284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d68a1-0c83-4da0-8f83-a7aeeab391e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:41.661698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d68a1-0c83-4da0-8f83-a7aeeab391e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:41.663740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d68a1-0c83-4da0-8f83-a7aeeab391e8 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:41.672507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1291/2556] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:43.185085 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d78f0-8b92-4913-a08f-16d937d5fa65 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:43.186960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-298d78f0-8b92-4913-a08f-16d937d5fa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:43.206483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d78f0-8b92-4913-a08f-16d937d5fa65 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:43.212251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0de5ba-1ac1-42dc-ad7b-93b52e964b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:43.213099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0de5ba-1ac1-42dc-ad7b-93b52e964b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:43.213387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0de5ba-1ac1-42dc-ad7b-93b52e964b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:43.214375 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0de5ba-1ac1-42dc-ad7b-93b52e964b2e admin admin] PATH_INFO: `/v3/users/ea2f6b804e164dbcb4f67926f23fdd59` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:43.215724 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0de5ba-1ac1-42dc-ad7b-93b52e964b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2f6b804e164dbcb4f67926f23fdd59)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:43.217676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0de5ba-1ac1-42dc-ad7b-93b52e964b2e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:43.507844 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f70a6a-830d-4319-a20f-554b21c63730 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:43.508340 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f70a6a-830d-4319-a20f-554b21c63730 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:43.508454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f70a6a-830d-4319-a20f-554b21c63730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:43.515795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:43.530380 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-10f70a6a-830d-4319-a20f-554b21c63730 None None] MFA Rules not processed for user `cc14169c8f7044c6815944af5bc658a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:43.532760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-10f70a6a-830d-4319-a20f-554b21c63730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:43.544484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1292/2557] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:43.547766 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f0de5ba-1ac1-42dc-ad7b-93b52e964b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2f6b804e164dbcb4f67926f23fdd59'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:43.562506 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f0de5ba-1ac1-42dc-ad7b-93b52e964b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2f6b804e164dbcb4f67926f23fdd59'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:43.568767 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:43.571616 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f0de5ba-1ac1-42dc-ad7b-93b52e964b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2f6b804e164dbcb4f67926f23fdd59'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:43.650087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f0de5ba-1ac1-42dc-ad7b-93b52e964b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2f6b804e164dbcb4f67926f23fdd59'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:43.650832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f0de5ba-1ac1-42dc-ad7b-93b52e964b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2f6b804e164dbcb4f67926f23fdd59'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:43.658453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1266/2558] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:12:43 2022] DELETE /identity/v3/users/ea2f6b804e164dbcb4f67926f23fdd59 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:43.935056 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f70a6a-830d-4319-a20f-554b21c63730 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:43.949696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f70a6a-830d-4319-a20f-554b21c63730 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:44.013087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:44.013755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:44.014038 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:44.014311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] PATH_INFO: `/v3/projects/b807736ef6c54be2b4715f32a4cfea57` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:44.015765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b807736ef6c54be2b4715f32a4cfea57)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:44.017680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:44.155426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b807736ef6c54be2b4715f32a4cfea57', 'request_id': 'req-0dbbbe2e-835a-4050-8415-4ed138645f0f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:44.162329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b807736ef6c54be2b4715f32a4cfea57', 'request_id': 'req-0dbbbe2e-835a-4050-8415-4ed138645f0f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:44.169404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b807736ef6c54be2b4715f32a4cfea57', 'request_id': 'req-0dbbbe2e-835a-4050-8415-4ed138645f0f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:44.254636 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] The token cache is being invalidate because project b807736ef6c54be2b4715f32a4cfea57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:12:44.254895 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:44.255540 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0dbbbe2e-835a-4050-8415-4ed138645f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:44.257106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1293/2559] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:12:43 2022] DELETE /identity/v3/projects/b807736ef6c54be2b4715f32a4cfea57 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:44.262208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a4c9f-1752-4bcc-a639-b5dcd1faddd2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:44.262740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a4c9f-1752-4bcc-a639-b5dcd1faddd2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:44.262842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a4c9f-1752-4bcc-a639-b5dcd1faddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:44.291485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-376a4c9f-1752-4bcc-a639-b5dcd1faddd2 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:44.292847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-376a4c9f-1752-4bcc-a639-b5dcd1faddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:44.360499 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1267/2560] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:44.368436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e09b5-b3b8-4b55-b6ca-312c4ffe8d3b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:44.368724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e09b5-b3b8-4b55-b6ca-312c4ffe8d3b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:44.369002 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e09b5-b3b8-4b55-b6ca-312c4ffe8d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:44.390168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c88e09b5-b3b8-4b55-b6ca-312c4ffe8d3b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:44.391290 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c88e09b5-b3b8-4b55-b6ca-312c4ffe8d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:44.400957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1294/2561] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:44.405467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a4c9f-1752-4bcc-a639-b5dcd1faddd2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:44.420554 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a4c9f-1752-4bcc-a639-b5dcd1faddd2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:44.425311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311b422-3488-4557-b8d3-58de60ea7575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:44.425917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311b422-3488-4557-b8d3-58de60ea7575 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:44.426209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311b422-3488-4557-b8d3-58de60ea7575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:44.426466 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311b422-3488-4557-b8d3-58de60ea7575 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:44.427138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311b422-3488-4557-b8d3-58de60ea7575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:12:44.427441 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311b422-3488-4557-b8d3-58de60ea7575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:44.429089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311b422-3488-4557-b8d3-58de60ea7575 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:44.440866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1268/2562] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:12:44.445572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e09b5-b3b8-4b55-b6ca-312c4ffe8d3b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:44.455736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e09b5-b3b8-4b55-b6ca-312c4ffe8d3b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:44.461011 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1141b-20fb-4bc7-8368-5bd8ecefde51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:44.461680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1141b-20fb-4bc7-8368-5bd8ecefde51 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:44.462091 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1141b-20fb-4bc7-8368-5bd8ecefde51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:44.462395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1141b-20fb-4bc7-8368-5bd8ecefde51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:44.463042 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1141b-20fb-4bc7-8368-5bd8ecefde51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:44.464964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1141b-20fb-4bc7-8368-5bd8ecefde51 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:44.601218 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1295/2563] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:12:44 2022] POST /identity/v3/projects => generated 403 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:12:44.605315 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311b422-3488-4557-b8d3-58de60ea7575 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:44.615984 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311b422-3488-4557-b8d3-58de60ea7575 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:44.686841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25578800-38a2-427f-bd5a-a119f6743fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:44.687460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25578800-38a2-427f-bd5a-a119f6743fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:44.687775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25578800-38a2-427f-bd5a-a119f6743fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:44.688203 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25578800-38a2-427f-bd5a-a119f6743fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:44.688782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25578800-38a2-427f-bd5a-a119f6743fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:44.691220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25578800-38a2-427f-bd5a-a119f6743fd6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:44.795676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1269/2564] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:12:44 2022] POST /identity/v3/users => generated 302 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:12:44.799846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1141b-20fb-4bc7-8368-5bd8ecefde51 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:44.815387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1141b-20fb-4bc7-8368-5bd8ecefde51 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:44.821207 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789eb07-9d18-4959-bc34-8a5fd94c7cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:44.821846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789eb07-9d18-4959-bc34-8a5fd94c7cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:44.822129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789eb07-9d18-4959-bc34-8a5fd94c7cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:44.822424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789eb07-9d18-4959-bc34-8a5fd94c7cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:44.823116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789eb07-9d18-4959-bc34-8a5fd94c7cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:44.824850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789eb07-9d18-4959-bc34-8a5fd94c7cb0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:44.840842 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1296/2565] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:44.845083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25578800-38a2-427f-bd5a-a119f6743fd6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:44.856421 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25578800-38a2-427f-bd5a-a119f6743fd6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:44.861596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32304118-cba9-44ba-ac4f-23da7e44987c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:44.865673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32304118-cba9-44ba-ac4f-23da7e44987c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:44.865673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32304118-cba9-44ba-ac4f-23da7e44987c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:44.865673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32304118-cba9-44ba-ac4f-23da7e44987c admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:44.865673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32304118-cba9-44ba-ac4f-23da7e44987c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:44.866133 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32304118-cba9-44ba-ac4f-23da7e44987c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:44.877930 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1270/2566] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:44.882604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789eb07-9d18-4959-bc34-8a5fd94c7cb0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:44.898539 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789eb07-9d18-4959-bc34-8a5fd94c7cb0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:44.903987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec3b40-7774-4979-9624-75973b6c1dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:44.904644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec3b40-7774-4979-9624-75973b6c1dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:44.904921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec3b40-7774-4979-9624-75973b6c1dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:44.905192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec3b40-7774-4979-9624-75973b6c1dd9 admin admin] PATH_INFO: `/v3/projects/45f62965f71b427ea6fc9cdf6d644775/users/13794ce2fda4463681050826bdd444a0/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:44.926940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec3b40-7774-4979-9624-75973b6c1dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f62965f71b427ea6fc9cdf6d644775, user_id=13794ce2fda4463681050826bdd444a0, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:44.930475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec3b40-7774-4979-9624-75973b6c1dd9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:45.005465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1297/2567] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:44 2022] PUT /identity/v3/projects/45f62965f71b427ea6fc9cdf6d644775/users/13794ce2fda4463681050826bdd444a0/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:45.010947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae72005-546e-47f4-be18-3cc2f0e1060a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:45.011261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae72005-546e-47f4-be18-3cc2f0e1060a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:45.011542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae72005-546e-47f4-be18-3cc2f0e1060a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:45.031338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8ae72005-546e-47f4-be18-3cc2f0e1060a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:45.032544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8ae72005-546e-47f4-be18-3cc2f0e1060a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:45.081353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:45.092969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1271/2568] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:12:45.103209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec3b40-7774-4979-9624-75973b6c1dd9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:45.104835 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a0ec3b40-7774-4979-9624-75973b6c1dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:45.132580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:45.164452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec3b40-7774-4979-9624-75973b6c1dd9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:45.169179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a0d70-6a4f-4097-ad33-1c59fde023a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:45.169963 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a0d70-6a4f-4097-ad33-1c59fde023a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:45.170245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a0d70-6a4f-4097-ad33-1c59fde023a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:45.170572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a0d70-6a4f-4097-ad33-1c59fde023a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:45.183393 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a0d70-6a4f-4097-ad33-1c59fde023a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:45.184878 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a0d70-6a4f-4097-ad33-1c59fde023a2 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:45.195894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1298/2569] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:46.728948 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3e0a1-f7c4-4a49-bd1c-a48c9ff0e2d3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:46.729498 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3e0a1-f7c4-4a49-bd1c-a48c9ff0e2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:46.729586 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3e0a1-f7c4-4a49-bd1c-a48c9ff0e2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:46.762953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-57d3e0a1-f7c4-4a49-bd1c-a48c9ff0e2d3 None None] MFA Rules not processed for user `13794ce2fda4463681050826bdd444a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:46.764342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-57d3e0a1-f7c4-4a49-bd1c-a48c9ff0e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:46.833176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1272/2570] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:46 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:46.838488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a0d70-6a4f-4097-ad33-1c59fde023a2 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:46.848069 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a0d70-6a4f-4097-ad33-1c59fde023a2 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:46.855846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f34c9-4071-41fe-b7e2-3768fc12d09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:46.856461 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f34c9-4071-41fe-b7e2-3768fc12d09a admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:46.856738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f34c9-4071-41fe-b7e2-3768fc12d09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:46.857016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f34c9-4071-41fe-b7e2-3768fc12d09a admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:46.857646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f34c9-4071-41fe-b7e2-3768fc12d09a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:46.859687 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f34c9-4071-41fe-b7e2-3768fc12d09a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:46.982801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1299/2571] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:12:46 2022] POST /identity/v3/projects => generated 403 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:12:46.986573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3e0a1-f7c4-4a49-bd1c-a48c9ff0e2d3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:46.997452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3e0a1-f7c4-4a49-bd1c-a48c9ff0e2d3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:47.054030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2999d-d49b-4890-b262-de8a211aa0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:47.054713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2999d-d49b-4890-b262-de8a211aa0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:47.055011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2999d-d49b-4890-b262-de8a211aa0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:47.055287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2999d-d49b-4890-b262-de8a211aa0d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:47.055908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2999d-d49b-4890-b262-de8a211aa0d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:47.057868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2999d-d49b-4890-b262-de8a211aa0d7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:47.184662 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1273/2572] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:12:46 2022] POST /identity/v3/users => generated 308 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:47.185941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f34c9-4071-41fe-b7e2-3768fc12d09a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:47.196881 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f34c9-4071-41fe-b7e2-3768fc12d09a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:47.202124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6573bcc-b731-4d53-83a5-f3a658776446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:47.202801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573bcc-b731-4d53-83a5-f3a658776446 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:47.203082 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573bcc-b731-4d53-83a5-f3a658776446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:47.203356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573bcc-b731-4d53-83a5-f3a658776446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:47.203998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6573bcc-b731-4d53-83a5-f3a658776446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:47.205571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6573bcc-b731-4d53-83a5-f3a658776446 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:47.216941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1300/2573] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:47.221529 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2999d-d49b-4890-b262-de8a211aa0d7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:47.233105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2999d-d49b-4890-b262-de8a211aa0d7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:47.237871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5e53-b91a-4fa7-959e-45cfcc5aa469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:47.238578 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5e53-b91a-4fa7-959e-45cfcc5aa469 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:47.238899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5e53-b91a-4fa7-959e-45cfcc5aa469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:47.239180 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5e53-b91a-4fa7-959e-45cfcc5aa469 admin admin] PATH_INFO: `/v3/projects/e475705a58144eb0ba2487821cc02c18/users/119d2d25cb0a4996984c4f9520c775d1/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:47.255185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c5e53-b91a-4fa7-959e-45cfcc5aa469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e475705a58144eb0ba2487821cc02c18, user_id=119d2d25cb0a4996984c4f9520c775d1, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:47.258455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c5e53-b91a-4fa7-959e-45cfcc5aa469 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:47.325288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1274/2574] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:47 2022] PUT /identity/v3/projects/e475705a58144eb0ba2487821cc02c18/users/119d2d25cb0a4996984c4f9520c775d1/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:47.329358 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6573bcc-b731-4d53-83a5-f3a658776446 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:47.339409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6573bcc-b731-4d53-83a5-f3a658776446 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:47.399148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d82517-c7e4-4b35-86cb-00e21a17a824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:47.399958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d82517-c7e4-4b35-86cb-00e21a17a824 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:47.400251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d82517-c7e4-4b35-86cb-00e21a17a824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:47.400532 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d82517-c7e4-4b35-86cb-00e21a17a824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:47.401186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d82517-c7e4-4b35-86cb-00e21a17a824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:47.402880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d82517-c7e4-4b35-86cb-00e21a17a824 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:47.413564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1301/2575] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:47.417714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5e53-b91a-4fa7-959e-45cfcc5aa469 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:47.431186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5e53-b91a-4fa7-959e-45cfcc5aa469 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:47.435971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26cc0e-8149-4283-81e7-f452d1ef0d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:47.436617 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26cc0e-8149-4283-81e7-f452d1ef0d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:47.436925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26cc0e-8149-4283-81e7-f452d1ef0d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:47.437251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26cc0e-8149-4283-81e7-f452d1ef0d65 admin admin] PATH_INFO: `/v3/projects/e475705a58144eb0ba2487821cc02c18/users/119d2d25cb0a4996984c4f9520c775d1/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:47.439154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26cc0e-8149-4283-81e7-f452d1ef0d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e475705a58144eb0ba2487821cc02c18, user_id=119d2d25cb0a4996984c4f9520c775d1, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:47.441826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26cc0e-8149-4283-81e7-f452d1ef0d65 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:47.453505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1275/2576] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:47 2022] PUT /identity/v3/projects/e475705a58144eb0ba2487821cc02c18/users/119d2d25cb0a4996984c4f9520c775d1/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:48.968260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756eb84-c2e2-4c7f-8d9f-2baa4308caeb None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:48.969668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756eb84-c2e2-4c7f-8d9f-2baa4308caeb None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:48.970150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756eb84-c2e2-4c7f-8d9f-2baa4308caeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:49.008740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8756eb84-c2e2-4c7f-8d9f-2baa4308caeb None None] MFA Rules not processed for user `119d2d25cb0a4996984c4f9520c775d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:49.010317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8756eb84-c2e2-4c7f-8d9f-2baa4308caeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:49.137718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1302/2577] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:48 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:50.479383 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f777ef7-a2a4-4cbe-9bc6-ea25d78e21d1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:50.479945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f777ef7-a2a4-4cbe-9bc6-ea25d78e21d1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:50.480092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f777ef7-a2a4-4cbe-9bc6-ea25d78e21d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:50.504721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8f777ef7-a2a4-4cbe-9bc6-ea25d78e21d1 None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:50.506176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8f777ef7-a2a4-4cbe-9bc6-ea25d78e21d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:50.546920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:50.578153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1276/2578] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:12:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:50.590533 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756eb84-c2e2-4c7f-8d9f-2baa4308caeb None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:50.592034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8756eb84-c2e2-4c7f-8d9f-2baa4308caeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:50.612748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:50.669563 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756eb84-c2e2-4c7f-8d9f-2baa4308caeb None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:50.674857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd429d04-328c-4172-877f-878637f35c79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:50.675415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429d04-328c-4172-877f-878637f35c79 service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:50.675696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429d04-328c-4172-877f-878637f35c79 service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:50.675970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429d04-328c-4172-877f-878637f35c79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:50.684566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd429d04-328c-4172-877f-878637f35c79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:50.686157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd429d04-328c-4172-877f-878637f35c79 service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:50.695337 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1303/2579] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:12:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:12:51.032575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f777ef7-a2a4-4cbe-9bc6-ea25d78e21d1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:51.034090 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8f777ef7-a2a4-4cbe-9bc6-ea25d78e21d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:51.129794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994d80e-eb34-4560-958b-06cc86dcd5d0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:51.130470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994d80e-eb34-4560-958b-06cc86dcd5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:51.131587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994d80e-eb34-4560-958b-06cc86dcd5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:51.177522 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f777ef7-a2a4-4cbe-9bc6-ea25d78e21d1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:51.184495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431c360-8232-4676-94ca-44cae4c4b507 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:51.185472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431c360-8232-4676-94ca-44cae4c4b507 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:51.185987 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431c360-8232-4676-94ca-44cae4c4b507 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:51.186455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431c360-8232-4676-94ca-44cae4c4b507 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:51.189151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f431c360-8232-4676-94ca-44cae4c4b507 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:51.204959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431c360-8232-4676-94ca-44cae4c4b507 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:51.206903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5994d80e-eb34-4560-958b-06cc86dcd5d0 None None] MFA Rules not processed for user `e9cd1b533aa8456d95e1bdbeac678b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:51.207155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431c360-8232-4676-94ca-44cae4c4b507 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:51.210971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5994d80e-eb34-4560-958b-06cc86dcd5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:51.220826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1277/2580] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:51.315422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431c360-8232-4676-94ca-44cae4c4b507 service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:51.335250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1304/2581] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:51.441061 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431c360-8232-4676-94ca-44cae4c4b507 service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:51.447228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:51.448146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:51.448429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:51.448704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] PATH_INFO: `/v3/users/13794ce2fda4463681050826bdd444a0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:51.450422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13794ce2fda4463681050826bdd444a0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:51.454159 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:51.719409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13794ce2fda4463681050826bdd444a0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:51.767247 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13794ce2fda4463681050826bdd444a0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:51.777431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13794ce2fda4463681050826bdd444a0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:51.783201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:51.858754 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13794ce2fda4463681050826bdd444a0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:51.860653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13794ce2fda4463681050826bdd444a0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:51.878542 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1278/2582] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:12:51 2022] DELETE /identity/v3/users/13794ce2fda4463681050826bdd444a0 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:12:52.224299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994d80e-eb34-4560-958b-06cc86dcd5d0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:52.257709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994d80e-eb34-4560-958b-06cc86dcd5d0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:52.380749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:52.382041 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:52.382317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:52.382587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] PATH_INFO: `/v3/projects/45f62965f71b427ea6fc9cdf6d644775` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:52.383846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f62965f71b427ea6fc9cdf6d644775)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:52.385974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:52.535229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f62965f71b427ea6fc9cdf6d644775', 'request_id': 'req-85ddf1db-6524-4e10-8e73-b8411c3f488d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:52.537706 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:52.552004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f62965f71b427ea6fc9cdf6d644775', 'request_id': 'req-85ddf1db-6524-4e10-8e73-b8411c3f488d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:52.558437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f62965f71b427ea6fc9cdf6d644775', 'request_id': 'req-85ddf1db-6524-4e10-8e73-b8411c3f488d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:52.632351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] The token cache is being invalidate because project 45f62965f71b427ea6fc9cdf6d644775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:12:52.632644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:52.633277 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:52.634781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1305/2583] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:12:52 2022] DELETE /identity/v3/projects/45f62965f71b427ea6fc9cdf6d644775 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:52.638759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:52.640161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:52.653810 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e343aa0-f00d-4966-8113-81db388364d5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:52.711399 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e40b1-adc7-4bb5-bb97-1d31ca9f8c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:52.712038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e40b1-adc7-4bb5-bb97-1d31ca9f8c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:52.712336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e40b1-adc7-4bb5-bb97-1d31ca9f8c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:52.712603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e40b1-adc7-4bb5-bb97-1d31ca9f8c2e admin admin] PATH_INFO: `/v3/users/119d2d25cb0a4996984c4f9520c775d1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:52.713740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e40b1-adc7-4bb5-bb97-1d31ca9f8c2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=119d2d25cb0a4996984c4f9520c775d1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:52.715726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e40b1-adc7-4bb5-bb97-1d31ca9f8c2e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:52.905040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-429e40b1-adc7-4bb5-bb97-1d31ca9f8c2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '119d2d25cb0a4996984c4f9520c775d1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:52.919150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-429e40b1-adc7-4bb5-bb97-1d31ca9f8c2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '119d2d25cb0a4996984c4f9520c775d1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:52.926809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-429e40b1-adc7-4bb5-bb97-1d31ca9f8c2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '119d2d25cb0a4996984c4f9520c775d1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:53.004061 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-429e40b1-adc7-4bb5-bb97-1d31ca9f8c2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '119d2d25cb0a4996984c4f9520c775d1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:53.004757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-429e40b1-adc7-4bb5-bb97-1d31ca9f8c2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '119d2d25cb0a4996984c4f9520c775d1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:53.014831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1279/2584] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:12:52 2022] DELETE /identity/v3/users/119d2d25cb0a4996984c4f9520c775d1 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:12:53.247343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:53.248924 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:53.275129 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddf1db-6524-4e10-8e73-b8411c3f488d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:53.364245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:53.365013 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:53.365822 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:53.366099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] PATH_INFO: `/v3/projects/e475705a58144eb0ba2487821cc02c18` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:53.367274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e475705a58144eb0ba2487821cc02c18)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:53.369257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:53.512806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e475705a58144eb0ba2487821cc02c18', 'request_id': 'req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:53.531353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e475705a58144eb0ba2487821cc02c18', 'request_id': 'req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:53.538215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e475705a58144eb0ba2487821cc02c18', 'request_id': 'req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:53.637089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] The token cache is being invalidate because project e475705a58144eb0ba2487821cc02c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:12:53.637386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:53.639118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ad3eebb5-1632-4ba2-afa8-c0ade8e9fa9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:53.643356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1306/2585] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:12:53 2022] DELETE /identity/v3/projects/e475705a58144eb0ba2487821cc02c18 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:53.649434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d8c0a-6f35-43b0-b52b-552ae6a20d0e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:53.649953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d8c0a-6f35-43b0-b52b-552ae6a20d0e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:53.650458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d8c0a-6f35-43b0-b52b-552ae6a20d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:53.680062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-655d8c0a-6f35-43b0-b52b-552ae6a20d0e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:53.681490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-655d8c0a-6f35-43b0-b52b-552ae6a20d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:53.834725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1280/2586] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:53.838870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5848bb1-992b-49ac-9c7a-c0f8cde880c2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:53.839162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5848bb1-992b-49ac-9c7a-c0f8cde880c2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:53.839443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5848bb1-992b-49ac-9c7a-c0f8cde880c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:53.860675 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a5848bb1-992b-49ac-9c7a-c0f8cde880c2 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:53.861900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a5848bb1-992b-49ac-9c7a-c0f8cde880c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:53.870549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1307/2587] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:53.875213 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d8c0a-6f35-43b0-b52b-552ae6a20d0e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:53.885196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d8c0a-6f35-43b0-b52b-552ae6a20d0e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:53.889996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212c324-ef4f-4867-9eff-03ba06ca8b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:53.890606 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212c324-ef4f-4867-9eff-03ba06ca8b98 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:53.890923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212c324-ef4f-4867-9eff-03ba06ca8b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:53.891201 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212c324-ef4f-4867-9eff-03ba06ca8b98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:53.891857 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212c324-ef4f-4867-9eff-03ba06ca8b98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:12:53.892167 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212c324-ef4f-4867-9eff-03ba06ca8b98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:53.893800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212c324-ef4f-4867-9eff-03ba06ca8b98 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:53.902011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:53.909043 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1281/2588] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:12:53.914062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5848bb1-992b-49ac-9c7a-c0f8cde880c2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:53.924204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5848bb1-992b-49ac-9c7a-c0f8cde880c2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:53.929629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed848b-f97e-43b6-8016-046500555ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:53.930395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed848b-f97e-43b6-8016-046500555ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:53.930784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed848b-f97e-43b6-8016-046500555ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:53.931089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed848b-f97e-43b6-8016-046500555ac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:53.931738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed848b-f97e-43b6-8016-046500555ac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:53.933886 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed848b-f97e-43b6-8016-046500555ac3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:54.035572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1308/2589] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:12:53 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:54.040101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212c324-ef4f-4867-9eff-03ba06ca8b98 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:54.054263 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212c324-ef4f-4867-9eff-03ba06ca8b98 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:54.122417 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c2e8c-8d57-499b-ba5d-c37949bcfd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:54.123075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2e8c-8d57-499b-ba5d-c37949bcfd12 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:54.123361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2e8c-8d57-499b-ba5d-c37949bcfd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:54.123655 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2e8c-8d57-499b-ba5d-c37949bcfd12 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:54.124896 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c2e8c-8d57-499b-ba5d-c37949bcfd12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:54.127095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c2e8c-8d57-499b-ba5d-c37949bcfd12 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:54.218575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1282/2590] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:12:54 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:12:54.222695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed848b-f97e-43b6-8016-046500555ac3 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:54.236221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed848b-f97e-43b6-8016-046500555ac3 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:54.241104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fab7c-73a6-411a-92cc-ff123191663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:54.241823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fab7c-73a6-411a-92cc-ff123191663b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:54.242106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fab7c-73a6-411a-92cc-ff123191663b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:54.242383 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fab7c-73a6-411a-92cc-ff123191663b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:54.243063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67fab7c-73a6-411a-92cc-ff123191663b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:54.244672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67fab7c-73a6-411a-92cc-ff123191663b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:54.256865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1309/2591] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:54.261136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c2e8c-8d57-499b-ba5d-c37949bcfd12 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:54.272363 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c2e8c-8d57-499b-ba5d-c37949bcfd12 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:54.277148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef08a76-1bed-4b2e-86aa-91133a87d153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:54.277762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef08a76-1bed-4b2e-86aa-91133a87d153 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:54.278045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef08a76-1bed-4b2e-86aa-91133a87d153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:54.278575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef08a76-1bed-4b2e-86aa-91133a87d153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:54.279252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef08a76-1bed-4b2e-86aa-91133a87d153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:54.280886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef08a76-1bed-4b2e-86aa-91133a87d153 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:54.297371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1283/2592] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:54.299704 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fab7c-73a6-411a-92cc-ff123191663b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:54.311347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fab7c-73a6-411a-92cc-ff123191663b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:54.317406 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f6aab-231c-4bee-a2ad-0e933ecec9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:54.318289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f6aab-231c-4bee-a2ad-0e933ecec9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:54.318799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f6aab-231c-4bee-a2ad-0e933ecec9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:54.319264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f6aab-231c-4bee-a2ad-0e933ecec9b1 admin admin] PATH_INFO: `/v3/projects/ae96ca8046d74425a9aff07d682a94ee/users/f9254f409edf49d6825578e36d252c54/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:54.338153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f6aab-231c-4bee-a2ad-0e933ecec9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae96ca8046d74425a9aff07d682a94ee, user_id=f9254f409edf49d6825578e36d252c54, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:54.341775 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f6aab-231c-4bee-a2ad-0e933ecec9b1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:54.440628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1310/2593] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:54 2022] PUT /identity/v3/projects/ae96ca8046d74425a9aff07d682a94ee/users/f9254f409edf49d6825578e36d252c54/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:54.455979 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819e9aa-3155-496a-afd8-ee95f9f16d25 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:54.456326 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819e9aa-3155-496a-afd8-ee95f9f16d25 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:54.456698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819e9aa-3155-496a-afd8-ee95f9f16d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:54.488093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5819e9aa-3155-496a-afd8-ee95f9f16d25 None None] MFA Rules not processed for user `f9254f409edf49d6825578e36d252c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:54.489953 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5819e9aa-3155-496a-afd8-ee95f9f16d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:54.563817 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1284/2594] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:54.565036 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f6aab-231c-4bee-a2ad-0e933ecec9b1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:54.571500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:54.631862 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f6aab-231c-4bee-a2ad-0e933ecec9b1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:54.637694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51fc1c-789b-483e-aaea-b8b5b6e2bd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:54.638362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51fc1c-789b-483e-aaea-b8b5b6e2bd8c admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:54.638647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51fc1c-789b-483e-aaea-b8b5b6e2bd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:54.638951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51fc1c-789b-483e-aaea-b8b5b6e2bd8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:54.639587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51fc1c-789b-483e-aaea-b8b5b6e2bd8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:54.641505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51fc1c-789b-483e-aaea-b8b5b6e2bd8c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:54.738255 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1311/2595] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:12:54 2022] POST /identity/v3/projects => generated 367 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:54.742552 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819e9aa-3155-496a-afd8-ee95f9f16d25 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:54.753746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819e9aa-3155-496a-afd8-ee95f9f16d25 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:54.844593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814d275-e68d-4220-857c-575297e96e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:54.845766 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814d275-e68d-4220-857c-575297e96e40 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:54.846344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814d275-e68d-4220-857c-575297e96e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:54.846741 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814d275-e68d-4220-857c-575297e96e40 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:54.847598 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814d275-e68d-4220-857c-575297e96e40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:54.850645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814d275-e68d-4220-857c-575297e96e40 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:54.874367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:55.022802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1285/2596] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:12:54 2022] POST /identity/v3/users => generated 290 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:12:55.026432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51fc1c-789b-483e-aaea-b8b5b6e2bd8c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:55.039571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51fc1c-789b-483e-aaea-b8b5b6e2bd8c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:55.044863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacce04-5bf6-41dc-92a2-a796d9ebf829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:55.045488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacce04-5bf6-41dc-92a2-a796d9ebf829 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:55.045770 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacce04-5bf6-41dc-92a2-a796d9ebf829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:55.046070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacce04-5bf6-41dc-92a2-a796d9ebf829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:55.046826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacce04-5bf6-41dc-92a2-a796d9ebf829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:55.048544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacce04-5bf6-41dc-92a2-a796d9ebf829 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:55.073589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1312/2597] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:55.078092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814d275-e68d-4220-857c-575297e96e40 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:55.099636 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814d275-e68d-4220-857c-575297e96e40 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:55.104966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f7d6a-f39d-47be-963f-2db3a5cd85fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:55.105654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f7d6a-f39d-47be-963f-2db3a5cd85fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:55.105935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f7d6a-f39d-47be-963f-2db3a5cd85fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:55.106211 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f7d6a-f39d-47be-963f-2db3a5cd85fb admin admin] PATH_INFO: `/v3/projects/a4e8d893eb79404bad32f97bd06df1da/users/fbaa2b5e9b5a4f3d9f2b25ae9a7f1262/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:55.124472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f7d6a-f39d-47be-963f-2db3a5cd85fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e8d893eb79404bad32f97bd06df1da, user_id=fbaa2b5e9b5a4f3d9f2b25ae9a7f1262, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:55.127208 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f7d6a-f39d-47be-963f-2db3a5cd85fb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:55.199652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1286/2598] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:55 2022] PUT /identity/v3/projects/a4e8d893eb79404bad32f97bd06df1da/users/fbaa2b5e9b5a4f3d9f2b25ae9a7f1262/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:55.203682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacce04-5bf6-41dc-92a2-a796d9ebf829 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:55.215162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacce04-5bf6-41dc-92a2-a796d9ebf829 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:55.290762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ec9c8-4cd0-40d8-8240-a7d39fe6b0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:55.291457 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ec9c8-4cd0-40d8-8240-a7d39fe6b0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:55.291663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ec9c8-4cd0-40d8-8240-a7d39fe6b0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:55.291940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ec9c8-4cd0-40d8-8240-a7d39fe6b0c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:55.292594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ec9c8-4cd0-40d8-8240-a7d39fe6b0c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:55.294436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ec9c8-4cd0-40d8-8240-a7d39fe6b0c9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:55.306860 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1313/2599] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:12:55.310881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f7d6a-f39d-47be-963f-2db3a5cd85fb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:55.325714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f7d6a-f39d-47be-963f-2db3a5cd85fb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:55.338299 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0d029-f15e-47f5-95c0-610297c6381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:55.339392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0d029-f15e-47f5-95c0-610297c6381d admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:55.340316 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0d029-f15e-47f5-95c0-610297c6381d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:55.340671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0d029-f15e-47f5-95c0-610297c6381d admin admin] PATH_INFO: `/v3/projects/a4e8d893eb79404bad32f97bd06df1da/users/fbaa2b5e9b5a4f3d9f2b25ae9a7f1262/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:55.342883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0d029-f15e-47f5-95c0-610297c6381d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e8d893eb79404bad32f97bd06df1da, user_id=fbaa2b5e9b5a4f3d9f2b25ae9a7f1262, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:55.346293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0d029-f15e-47f5-95c0-610297c6381d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:55.359365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1287/2600] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:12:55 2022] PUT /identity/v3/projects/a4e8d893eb79404bad32f97bd06df1da/users/fbaa2b5e9b5a4f3d9f2b25ae9a7f1262/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:55.369934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32064f59-07c7-46ca-963e-03a51f5b9c3c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:55.370257 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32064f59-07c7-46ca-963e-03a51f5b9c3c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:55.370498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32064f59-07c7-46ca-963e-03a51f5b9c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:55.431162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-32064f59-07c7-46ca-963e-03a51f5b9c3c None None] MFA Rules not processed for user `fbaa2b5e9b5a4f3d9f2b25ae9a7f1262`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:55.432395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-32064f59-07c7-46ca-963e-03a51f5b9c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:55.517363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:12:55.526470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1314/2601] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:55.539819 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0d029-f15e-47f5-95c0-610297c6381d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:55.541785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-70c0d029-f15e-47f5-95c0-610297c6381d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:55.609947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0d029-f15e-47f5-95c0-610297c6381d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:55.614136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668436a-a4e2-4957-a0f6-bf921c3336d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:55.614732 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668436a-a4e2-4957-a0f6-bf921c3336d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:55.615021 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668436a-a4e2-4957-a0f6-bf921c3336d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:55.615880 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668436a-a4e2-4957-a0f6-bf921c3336d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:55.625738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668436a-a4e2-4957-a0f6-bf921c3336d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:55.627289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668436a-a4e2-4957-a0f6-bf921c3336d0 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:55.638433 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1288/2602] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:12:59.147289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32064f59-07c7-46ca-963e-03a51f5b9c3c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:59.214701 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32064f59-07c7-46ca-963e-03a51f5b9c3c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:59.219219 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:59.220177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:59.220306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:59.220604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] PATH_INFO: `/v3/users/f9254f409edf49d6825578e36d252c54` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:59.221890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9254f409edf49d6825578e36d252c54)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:59.223946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:59.364408 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9254f409edf49d6825578e36d252c54'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:59.395111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9254f409edf49d6825578e36d252c54'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:59.402665 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9254f409edf49d6825578e36d252c54'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:59.467422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9254f409edf49d6825578e36d252c54'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:59.467973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9254f409edf49d6825578e36d252c54'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:12:59.474341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1315/2603] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:12:59 2022] DELETE /identity/v3/users/f9254f409edf49d6825578e36d252c54 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:12:59.479374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a9eb3-9a10-4519-aabc-038c057503f7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:59.484392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a9eb3-9a10-4519-aabc-038c057503f7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:59.484392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a9eb3-9a10-4519-aabc-038c057503f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:59.499785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4e3a9eb3-9a10-4519-aabc-038c057503f7 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:12:59.501223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4e3a9eb3-9a10-4519-aabc-038c057503f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:59.565003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1289/2604] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:12:59.572740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:12:59.574153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:12:59.633978 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc814083-bae7-47e4-8c8b-736593ee411d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:12:59.637679 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883713a5-e5b6-4899-bcfc-48b124e84a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:12:59.638284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883713a5-e5b6-4899-bcfc-48b124e84a86 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:12:59.638567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883713a5-e5b6-4899-bcfc-48b124e84a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:12:59.638875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883713a5-e5b6-4899-bcfc-48b124e84a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:12:59.648386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883713a5-e5b6-4899-bcfc-48b124e84a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:12:59.649802 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883713a5-e5b6-4899-bcfc-48b124e84a86 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:12:59.659783 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1316/2605] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:13:00.159794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a9eb3-9a10-4519-aabc-038c057503f7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:00.161320 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4e3a9eb3-9a10-4519-aabc-038c057503f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:00.180844 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a9eb3-9a10-4519-aabc-038c057503f7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:00.190401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:00.191154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:00.191347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:00.191627 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] PATH_INFO: `/v3/projects/ae96ca8046d74425a9aff07d682a94ee` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:00.193972 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae96ca8046d74425a9aff07d682a94ee)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:00.195741 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:00.301573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae96ca8046d74425a9aff07d682a94ee', 'request_id': 'req-417e7652-6dc3-4e67-9148-7295d736d6ac'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:00.308351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae96ca8046d74425a9aff07d682a94ee', 'request_id': 'req-417e7652-6dc3-4e67-9148-7295d736d6ac'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:00.310252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:00.314184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae96ca8046d74425a9aff07d682a94ee', 'request_id': 'req-417e7652-6dc3-4e67-9148-7295d736d6ac'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:00.374472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] The token cache is being invalidate because project ae96ca8046d74425a9aff07d682a94ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:13:00.374796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:00.386031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:00.386031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1290/2606] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:13:00 2022] DELETE /identity/v3/projects/ae96ca8046d74425a9aff07d682a94ee => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:00.386260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883713a5-e5b6-4899-bcfc-48b124e84a86 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:00.391548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883713a5-e5b6-4899-bcfc-48b124e84a86 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:00.412633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:00.452029 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b38320-87de-4797-9bdb-47800d145d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:00.452661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b38320-87de-4797-9bdb-47800d145d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:00.452938 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b38320-87de-4797-9bdb-47800d145d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:00.453241 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b38320-87de-4797-9bdb-47800d145d11 admin admin] PATH_INFO: `/v3/users/fbaa2b5e9b5a4f3d9f2b25ae9a7f1262` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:00.454347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b38320-87de-4797-9bdb-47800d145d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbaa2b5e9b5a4f3d9f2b25ae9a7f1262)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:00.456331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b38320-87de-4797-9bdb-47800d145d11 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:00.610580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e7b38320-87de-4797-9bdb-47800d145d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbaa2b5e9b5a4f3d9f2b25ae9a7f1262'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:00.622984 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e7b38320-87de-4797-9bdb-47800d145d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbaa2b5e9b5a4f3d9f2b25ae9a7f1262'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:00.629922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e7b38320-87de-4797-9bdb-47800d145d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbaa2b5e9b5a4f3d9f2b25ae9a7f1262'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:00.712633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e7b38320-87de-4797-9bdb-47800d145d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbaa2b5e9b5a4f3d9f2b25ae9a7f1262'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:00.713261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e7b38320-87de-4797-9bdb-47800d145d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbaa2b5e9b5a4f3d9f2b25ae9a7f1262'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:00.719577 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1317/2607] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:13:00 2022] DELETE /identity/v3/users/fbaa2b5e9b5a4f3d9f2b25ae9a7f1262 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:01.164351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:01.165469 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:01.183258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e7652-6dc3-4e67-9148-7295d736d6ac admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:01.250750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:01.250750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:01.250750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:01.250750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] PATH_INFO: `/v3/projects/a4e8d893eb79404bad32f97bd06df1da` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:01.251474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e8d893eb79404bad32f97bd06df1da)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:01.252413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:01.408676 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e8d893eb79404bad32f97bd06df1da', 'request_id': 'req-1a2b2492-7b6d-47d1-bda2-c572ff438a09'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:01.416862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e8d893eb79404bad32f97bd06df1da', 'request_id': 'req-1a2b2492-7b6d-47d1-bda2-c572ff438a09'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:01.423034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e8d893eb79404bad32f97bd06df1da', 'request_id': 'req-1a2b2492-7b6d-47d1-bda2-c572ff438a09'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:01.496479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] The token cache is being invalidate because project a4e8d893eb79404bad32f97bd06df1da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:13:01.496784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:01.497447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1a2b2492-7b6d-47d1-bda2-c572ff438a09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:01.499218 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1291/2608] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:13:01 2022] DELETE /identity/v3/projects/a4e8d893eb79404bad32f97bd06df1da => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:01.504347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096c420-469d-42a4-ac91-ddfcb79fa20a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:01.504781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096c420-469d-42a4-ac91-ddfcb79fa20a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:01.505023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096c420-469d-42a4-ac91-ddfcb79fa20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:01.525970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d096c420-469d-42a4-ac91-ddfcb79fa20a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:01.527313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d096c420-469d-42a4-ac91-ddfcb79fa20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:01.590973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1318/2609] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:01.599961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4ea0d-ebd0-4c9f-9fd6-fe622b68dbc8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:01.600318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4ea0d-ebd0-4c9f-9fd6-fe622b68dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:01.601169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4ea0d-ebd0-4c9f-9fd6-fe622b68dbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:01.621942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-07b4ea0d-ebd0-4c9f-9fd6-fe622b68dbc8 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:01.623176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-07b4ea0d-ebd0-4c9f-9fd6-fe622b68dbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:01.630971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1292/2610] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:01.635181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096c420-469d-42a4-ac91-ddfcb79fa20a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:01.645294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096c420-469d-42a4-ac91-ddfcb79fa20a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:01.649856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc43e19-00d4-4a16-ad66-95cde1a0da40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:01.650450 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc43e19-00d4-4a16-ad66-95cde1a0da40 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:01.650759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc43e19-00d4-4a16-ad66-95cde1a0da40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:01.651032 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc43e19-00d4-4a16-ad66-95cde1a0da40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:01.651667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc43e19-00d4-4a16-ad66-95cde1a0da40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:13:01.652298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc43e19-00d4-4a16-ad66-95cde1a0da40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:01.653803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc43e19-00d4-4a16-ad66-95cde1a0da40 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:01.666211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1319/2611] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:13:01.670454 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4ea0d-ebd0-4c9f-9fd6-fe622b68dbc8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:01.679749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4ea0d-ebd0-4c9f-9fd6-fe622b68dbc8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:01.684456 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358eb83-b54b-4bbd-a17e-4800307f735d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:01.685160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358eb83-b54b-4bbd-a17e-4800307f735d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:01.685449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358eb83-b54b-4bbd-a17e-4800307f735d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:01.685726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358eb83-b54b-4bbd-a17e-4800307f735d admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:01.686373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358eb83-b54b-4bbd-a17e-4800307f735d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:01.688336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358eb83-b54b-4bbd-a17e-4800307f735d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:01.760185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1293/2612] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:13:01 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:13:01.764462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc43e19-00d4-4a16-ad66-95cde1a0da40 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:01.775789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc43e19-00d4-4a16-ad66-95cde1a0da40 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:01.802684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:01.832649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae86e1-f9ce-430c-aa22-8dad6119704b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:01.833252 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae86e1-f9ce-430c-aa22-8dad6119704b admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:01.833535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae86e1-f9ce-430c-aa22-8dad6119704b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:01.833806 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae86e1-f9ce-430c-aa22-8dad6119704b admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:01.834486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae86e1-f9ce-430c-aa22-8dad6119704b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:01.837003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae86e1-f9ce-430c-aa22-8dad6119704b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:01.919114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1320/2613] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:13:01 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:13:01.923119 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358eb83-b54b-4bbd-a17e-4800307f735d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:01.933354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358eb83-b54b-4bbd-a17e-4800307f735d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:01.937904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb705a4-a14d-4560-8510-de701747e248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:01.938492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb705a4-a14d-4560-8510-de701747e248 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:01.938789 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb705a4-a14d-4560-8510-de701747e248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:01.939058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb705a4-a14d-4560-8510-de701747e248 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:01.939690 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb705a4-a14d-4560-8510-de701747e248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:01.941314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb705a4-a14d-4560-8510-de701747e248 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:01.952061 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1294/2614] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:01.956245 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae86e1-f9ce-430c-aa22-8dad6119704b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:01.966803 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae86e1-f9ce-430c-aa22-8dad6119704b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:01.971363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b4f32-4f08-4f03-b0e8-df530213c7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:01.971949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b4f32-4f08-4f03-b0e8-df530213c7fa admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:01.972232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b4f32-4f08-4f03-b0e8-df530213c7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:01.972515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b4f32-4f08-4f03-b0e8-df530213c7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:01.973155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b4f32-4f08-4f03-b0e8-df530213c7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:01.974750 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b4f32-4f08-4f03-b0e8-df530213c7fa admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:01.984459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1321/2615] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:01.988413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb705a4-a14d-4560-8510-de701747e248 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:01.998056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb705a4-a14d-4560-8510-de701747e248 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:02.002567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df71307-0b4e-4888-9983-6f63d8ed0ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:02.003240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df71307-0b4e-4888-9983-6f63d8ed0ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:02.003515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df71307-0b4e-4888-9983-6f63d8ed0ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:02.003790 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df71307-0b4e-4888-9983-6f63d8ed0ba6 admin admin] PATH_INFO: `/v3/projects/15111e6724dd45dabf3d68055789823b/users/688b1f101ea04f27ba9f0aa356afb4e9/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:02.018283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df71307-0b4e-4888-9983-6f63d8ed0ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15111e6724dd45dabf3d68055789823b, user_id=688b1f101ea04f27ba9f0aa356afb4e9, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:02.020969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df71307-0b4e-4888-9983-6f63d8ed0ba6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:02.112217 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1295/2616] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:13:01 2022] PUT /identity/v3/projects/15111e6724dd45dabf3d68055789823b/users/688b1f101ea04f27ba9f0aa356afb4e9/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:02.122224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581be9c-0cbc-44e5-ae32-a1c99905344c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:02.122508 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581be9c-0cbc-44e5-ae32-a1c99905344c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:02.122888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581be9c-0cbc-44e5-ae32-a1c99905344c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:02.152490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d581be9c-0cbc-44e5-ae32-a1c99905344c None None] MFA Rules not processed for user `688b1f101ea04f27ba9f0aa356afb4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:02.153986 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d581be9c-0cbc-44e5-ae32-a1c99905344c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:02.240298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1322/2617] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:02.245228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df71307-0b4e-4888-9983-6f63d8ed0ba6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:02.260189 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:02.312659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df71307-0b4e-4888-9983-6f63d8ed0ba6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:02.317784 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130825d-3091-4456-a106-54e21b78de5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:02.318487 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130825d-3091-4456-a106-54e21b78de5a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:02.318824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130825d-3091-4456-a106-54e21b78de5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:02.319108 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130825d-3091-4456-a106-54e21b78de5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:02.320386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130825d-3091-4456-a106-54e21b78de5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:02.322373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130825d-3091-4456-a106-54e21b78de5a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:02.412544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1296/2618] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:13:02 2022] POST /identity/v3/projects => generated 387 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:13:02.416429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581be9c-0cbc-44e5-ae32-a1c99905344c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:02.427935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581be9c-0cbc-44e5-ae32-a1c99905344c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:02.490184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821cbfe-589d-4626-b7f8-d9ee6bfa5c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:02.490823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821cbfe-589d-4626-b7f8-d9ee6bfa5c6c admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:02.491104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821cbfe-589d-4626-b7f8-d9ee6bfa5c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:02.491996 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821cbfe-589d-4626-b7f8-d9ee6bfa5c6c admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:02.492675 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821cbfe-589d-4626-b7f8-d9ee6bfa5c6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:02.494589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821cbfe-589d-4626-b7f8-d9ee6bfa5c6c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:02.601601 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1323/2619] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:13:02 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:13:02.605170 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130825d-3091-4456-a106-54e21b78de5a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:02.613937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130825d-3091-4456-a106-54e21b78de5a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:02.620185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d6d19-f664-4451-a625-ac24ee0c900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:02.621334 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d6d19-f664-4451-a625-ac24ee0c900d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:02.621623 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d6d19-f664-4451-a625-ac24ee0c900d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:02.621976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d6d19-f664-4451-a625-ac24ee0c900d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:02.622860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d6d19-f664-4451-a625-ac24ee0c900d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:02.624210 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d6d19-f664-4451-a625-ac24ee0c900d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:02.638425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1297/2620] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:02.642375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821cbfe-589d-4626-b7f8-d9ee6bfa5c6c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:02.658114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821cbfe-589d-4626-b7f8-d9ee6bfa5c6c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:02.667033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a5f62-7154-49c3-8546-1e6538d1e16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:02.667829 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a5f62-7154-49c3-8546-1e6538d1e16b admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:02.668122 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a5f62-7154-49c3-8546-1e6538d1e16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:02.668552 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a5f62-7154-49c3-8546-1e6538d1e16b admin admin] PATH_INFO: `/v3/projects/141f2ab804d745e2a84800bca2e4f905/users/1900d73138e54bc89817d1753f7474be/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:02.701204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a5f62-7154-49c3-8546-1e6538d1e16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=141f2ab804d745e2a84800bca2e4f905, user_id=1900d73138e54bc89817d1753f7474be, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:02.704089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a5f62-7154-49c3-8546-1e6538d1e16b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:02.770427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1324/2621] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:13:02 2022] PUT /identity/v3/projects/141f2ab804d745e2a84800bca2e4f905/users/1900d73138e54bc89817d1753f7474be/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:13:02.774728 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d6d19-f664-4451-a625-ac24ee0c900d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:02.785445 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d6d19-f664-4451-a625-ac24ee0c900d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:02.844448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fdadd-a539-4b51-8581-7ad2b10e16e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:02.845105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fdadd-a539-4b51-8581-7ad2b10e16e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:02.845392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fdadd-a539-4b51-8581-7ad2b10e16e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:02.846047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fdadd-a539-4b51-8581-7ad2b10e16e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:02.846344 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fdadd-a539-4b51-8581-7ad2b10e16e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:02.848012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fdadd-a539-4b51-8581-7ad2b10e16e1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:02.859047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1298/2622] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:02.863155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a5f62-7154-49c3-8546-1e6538d1e16b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:02.870572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:02.874484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a5f62-7154-49c3-8546-1e6538d1e16b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:02.879131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba9590-958e-4247-b7a5-8c8da50b35cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:02.879797 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba9590-958e-4247-b7a5-8c8da50b35cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:02.880081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba9590-958e-4247-b7a5-8c8da50b35cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:02.880362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba9590-958e-4247-b7a5-8c8da50b35cd admin admin] PATH_INFO: `/v3/projects/141f2ab804d745e2a84800bca2e4f905/users/1900d73138e54bc89817d1753f7474be/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:02.882341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba9590-958e-4247-b7a5-8c8da50b35cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=141f2ab804d745e2a84800bca2e4f905, user_id=1900d73138e54bc89817d1753f7474be, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:02.885109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba9590-958e-4247-b7a5-8c8da50b35cd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:02.895627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1325/2623] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:13:02 2022] PUT /identity/v3/projects/141f2ab804d745e2a84800bca2e4f905/users/1900d73138e54bc89817d1753f7474be/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:02.905864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8330a-e8e3-4b11-90cf-6fabdc22a86b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:02.906152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8330a-e8e3-4b11-90cf-6fabdc22a86b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:02.908197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8330a-e8e3-4b11-90cf-6fabdc22a86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:02.934564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a1b8330a-e8e3-4b11-90cf-6fabdc22a86b None None] MFA Rules not processed for user `1900d73138e54bc89817d1753f7474be`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:02.935761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a1b8330a-e8e3-4b11-90cf-6fabdc22a86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:03.008519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1299/2624] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:03.019661 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba9590-958e-4247-b7a5-8c8da50b35cd admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:03.021446 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-83ba9590-958e-4247-b7a5-8c8da50b35cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:03.080627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba9590-958e-4247-b7a5-8c8da50b35cd admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:03.084202 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7dcbc-8165-4d80-b7fe-ccc6a7f2c67e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:03.084795 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7dcbc-8165-4d80-b7fe-ccc6a7f2c67e service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:03.085113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7dcbc-8165-4d80-b7fe-ccc6a7f2c67e service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:03.085385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7dcbc-8165-4d80-b7fe-ccc6a7f2c67e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:03.094289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7dcbc-8165-4d80-b7fe-ccc6a7f2c67e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:03.095739 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7dcbc-8165-4d80-b7fe-ccc6a7f2c67e service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:03.105070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1326/2625] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:13:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:13:03.134374 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8330a-e8e3-4b11-90cf-6fabdc22a86b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:03.147303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:03.201829 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8330a-e8e3-4b11-90cf-6fabdc22a86b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:03.206614 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:03.207357 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:03.207761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:03.208044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] PATH_INFO: `/v3/users/688b1f101ea04f27ba9f0aa356afb4e9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:03.209300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=688b1f101ea04f27ba9f0aa356afb4e9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:03.211279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:03.451702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '688b1f101ea04f27ba9f0aa356afb4e9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:03.463507 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '688b1f101ea04f27ba9f0aa356afb4e9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:03.471721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '688b1f101ea04f27ba9f0aa356afb4e9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:03.536077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '688b1f101ea04f27ba9f0aa356afb4e9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:03.536707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '688b1f101ea04f27ba9f0aa356afb4e9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:03.543933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1300/2626] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:13:03 2022] DELETE /identity/v3/users/688b1f101ea04f27ba9f0aa356afb4e9 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:03.548449 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691029-7ccc-4337-b5c7-7d3346d140c0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:03.548737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691029-7ccc-4337-b5c7-7d3346d140c0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:03.549012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691029-7ccc-4337-b5c7-7d3346d140c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:03.568626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b6691029-7ccc-4337-b5c7-7d3346d140c0 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:03.569872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b6691029-7ccc-4337-b5c7-7d3346d140c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:03.691111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1327/2627] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:03.713671 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:03.715520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:03.810087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d686a-1a56-49d5-8f5c-a1fc1b3da657 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:03.815505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3812f-5327-4eb7-b058-809692ebdb44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:03.816308 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3812f-5327-4eb7-b058-809692ebdb44 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:03.817605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3812f-5327-4eb7-b058-809692ebdb44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:03.817605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3812f-5327-4eb7-b058-809692ebdb44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:03.830435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3812f-5327-4eb7-b058-809692ebdb44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:03.831954 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3812f-5327-4eb7-b058-809692ebdb44 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:03.852546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1301/2628] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:13:04.824387 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6691029-7ccc-4337-b5c7-7d3346d140c0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:04.826127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b6691029-7ccc-4337-b5c7-7d3346d140c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:04.840427 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6691029-7ccc-4337-b5c7-7d3346d140c0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:04.845185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:04.845902 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:04.846283 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:04.847095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] PATH_INFO: `/v3/projects/15111e6724dd45dabf3d68055789823b` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:04.847853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15111e6724dd45dabf3d68055789823b)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:04.850109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:04.956579 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15111e6724dd45dabf3d68055789823b', 'request_id': 'req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:04.964527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15111e6724dd45dabf3d68055789823b', 'request_id': 'req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:04.971747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15111e6724dd45dabf3d68055789823b', 'request_id': 'req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:05.067489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] The token cache is being invalidate because project 15111e6724dd45dabf3d68055789823b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:13:05.069952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:05.069952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:05.072264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1328/2629] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:13:04 2022] DELETE /identity/v3/projects/15111e6724dd45dabf3d68055789823b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:05.077030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3812f-5327-4eb7-b058-809692ebdb44 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:05.090171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3812f-5327-4eb7-b058-809692ebdb44 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:05.120389 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:05.152650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5dfee-5423-4ed0-bcd0-82900f78ee85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:05.153812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5dfee-5423-4ed0-bcd0-82900f78ee85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:05.153930 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5dfee-5423-4ed0-bcd0-82900f78ee85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:05.154199 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5dfee-5423-4ed0-bcd0-82900f78ee85 admin admin] PATH_INFO: `/v3/users/1900d73138e54bc89817d1753f7474be` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:05.155385 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5dfee-5423-4ed0-bcd0-82900f78ee85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1900d73138e54bc89817d1753f7474be)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:05.157352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5dfee-5423-4ed0-bcd0-82900f78ee85 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:05.380495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-aac5dfee-5423-4ed0-bcd0-82900f78ee85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1900d73138e54bc89817d1753f7474be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:05.416969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-aac5dfee-5423-4ed0-bcd0-82900f78ee85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1900d73138e54bc89817d1753f7474be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:05.426764 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-aac5dfee-5423-4ed0-bcd0-82900f78ee85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1900d73138e54bc89817d1753f7474be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:05.503792 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-aac5dfee-5423-4ed0-bcd0-82900f78ee85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1900d73138e54bc89817d1753f7474be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:05.504645 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-aac5dfee-5423-4ed0-bcd0-82900f78ee85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1900d73138e54bc89817d1753f7474be'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:05.512067 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1302/2630] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:13:05 2022] DELETE /identity/v3/users/1900d73138e54bc89817d1753f7474be => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:06.096330 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:06.101308 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:06.111977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3221a-a6b0-43d1-87ec-8a9bd45b2a34 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:06.130669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:06.178046 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:06.178695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:06.178987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:06.179264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] PATH_INFO: `/v3/projects/141f2ab804d745e2a84800bca2e4f905` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:06.180530 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=141f2ab804d745e2a84800bca2e4f905)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:06.182490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:06.318571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141f2ab804d745e2a84800bca2e4f905', 'request_id': 'req-a2ee86fd-7b56-4748-844c-10f20520dcf3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:06.325054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141f2ab804d745e2a84800bca2e4f905', 'request_id': 'req-a2ee86fd-7b56-4748-844c-10f20520dcf3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:06.330626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141f2ab804d745e2a84800bca2e4f905', 'request_id': 'req-a2ee86fd-7b56-4748-844c-10f20520dcf3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:06.437663 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] The token cache is being invalidate because project 141f2ab804d745e2a84800bca2e4f905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:13:06.437981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:06.438777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a2ee86fd-7b56-4748-844c-10f20520dcf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:06.440359 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1329/2631] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:13:06 2022] DELETE /identity/v3/projects/141f2ab804d745e2a84800bca2e4f905 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:06.446495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdd7c0-6eb7-4332-be9f-61cd5611dfb9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:06.447081 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdd7c0-6eb7-4332-be9f-61cd5611dfb9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:06.447196 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdd7c0-6eb7-4332-be9f-61cd5611dfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:06.467195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bbbdd7c0-6eb7-4332-be9f-61cd5611dfb9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:06.469472 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bbbdd7c0-6eb7-4332-be9f-61cd5611dfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:06.539812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1303/2632] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:13:06.548398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36486688-de34-4b6c-ada9-ddbb4df61c8e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:06.548690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36486688-de34-4b6c-ada9-ddbb4df61c8e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:06.548972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36486688-de34-4b6c-ada9-ddbb4df61c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:06.587894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-36486688-de34-4b6c-ada9-ddbb4df61c8e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:06.589294 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-36486688-de34-4b6c-ada9-ddbb4df61c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:06.597696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1330/2633] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:06.601834 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdd7c0-6eb7-4332-be9f-61cd5611dfb9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:06.627878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdd7c0-6eb7-4332-be9f-61cd5611dfb9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:06.640306 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e6a42-1c6b-44b0-ba20-998af6381f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:06.641252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e6a42-1c6b-44b0-ba20-998af6381f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:06.643093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e6a42-1c6b-44b0-ba20-998af6381f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:06.643379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e6a42-1c6b-44b0-ba20-998af6381f9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:06.644111 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e6a42-1c6b-44b0-ba20-998af6381f9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:13:06.644366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e6a42-1c6b-44b0-ba20-998af6381f9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:06.646017 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e6a42-1c6b-44b0-ba20-998af6381f9a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:06.662697 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1304/2634] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:13:06.664085 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36486688-de34-4b6c-ada9-ddbb4df61c8e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:06.674653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36486688-de34-4b6c-ada9-ddbb4df61c8e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:06.679765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32593b14-e67d-4ab4-ad2d-2700eb83aabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:06.680644 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32593b14-e67d-4ab4-ad2d-2700eb83aabc admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:06.683686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32593b14-e67d-4ab4-ad2d-2700eb83aabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:06.683686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32593b14-e67d-4ab4-ad2d-2700eb83aabc admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:06.691268 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32593b14-e67d-4ab4-ad2d-2700eb83aabc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:06.693198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32593b14-e67d-4ab4-ad2d-2700eb83aabc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:06.806233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1331/2635] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:13:06 2022] POST /identity/v3/projects => generated 389 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:13:06.810425 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e6a42-1c6b-44b0-ba20-998af6381f9a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:06.821133 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e6a42-1c6b-44b0-ba20-998af6381f9a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:06.853870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:06.880064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab7e6c-24ec-4745-9277-05771fad0f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:06.880592 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab7e6c-24ec-4745-9277-05771fad0f1e admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:06.880871 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab7e6c-24ec-4745-9277-05771fad0f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:06.881144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab7e6c-24ec-4745-9277-05771fad0f1e admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:06.881779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab7e6c-24ec-4745-9277-05771fad0f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:06.884356 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab7e6c-24ec-4745-9277-05771fad0f1e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:07.041116 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1305/2636] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:13:06 2022] POST /identity/v3/users => generated 295 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:13:07.042411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32593b14-e67d-4ab4-ad2d-2700eb83aabc admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:07.052852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32593b14-e67d-4ab4-ad2d-2700eb83aabc admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:07.058162 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02968476-9b4b-4aea-a4b7-be82c03bc7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:07.059826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02968476-9b4b-4aea-a4b7-be82c03bc7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:07.059826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02968476-9b4b-4aea-a4b7-be82c03bc7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:07.059826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02968476-9b4b-4aea-a4b7-be82c03bc7f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:07.060227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02968476-9b4b-4aea-a4b7-be82c03bc7f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:07.061687 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02968476-9b4b-4aea-a4b7-be82c03bc7f1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:07.072735 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1332/2637] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:07.077402 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab7e6c-24ec-4745-9277-05771fad0f1e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:07.088698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab7e6c-24ec-4745-9277-05771fad0f1e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:07.093380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5d3bc-4fa5-4a27-8299-5cfc1db3a737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:07.094014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5d3bc-4fa5-4a27-8299-5cfc1db3a737 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:07.094293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5d3bc-4fa5-4a27-8299-5cfc1db3a737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:07.094573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5d3bc-4fa5-4a27-8299-5cfc1db3a737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:07.095249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5d3bc-4fa5-4a27-8299-5cfc1db3a737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:07.096833 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5d3bc-4fa5-4a27-8299-5cfc1db3a737 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:07.109256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1306/2638] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:07.111397 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02968476-9b4b-4aea-a4b7-be82c03bc7f1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:07.121778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02968476-9b4b-4aea-a4b7-be82c03bc7f1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:07.126281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577dc65-24d7-4035-8f82-33100aa97dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:07.126944 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577dc65-24d7-4035-8f82-33100aa97dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:07.127225 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577dc65-24d7-4035-8f82-33100aa97dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:07.127498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577dc65-24d7-4035-8f82-33100aa97dc9 admin admin] PATH_INFO: `/v3/projects/940c02c35da34b9081d98ae0aeddfbdd/users/2e3546dc78074fb68a12fc8a9d920be1/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:07.145859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577dc65-24d7-4035-8f82-33100aa97dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=940c02c35da34b9081d98ae0aeddfbdd, user_id=2e3546dc78074fb68a12fc8a9d920be1, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:07.146784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577dc65-24d7-4035-8f82-33100aa97dc9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:07.183431 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40e865-4cf0-4920-875a-813d894166ca None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:07.183794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40e865-4cf0-4920-875a-813d894166ca None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:07.184279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40e865-4cf0-4920-875a-813d894166ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:07.206083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bb40e865-4cf0-4920-875a-813d894166ca None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:07.207946 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bb40e865-4cf0-4920-875a-813d894166ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:07.254783 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1333/2639] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:13:07 2022] PUT /identity/v3/projects/940c02c35da34b9081d98ae0aeddfbdd/users/2e3546dc78074fb68a12fc8a9d920be1/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:07.256400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0709af8-4acf-47e1-af64-c93cb361f273 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:07.256685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0709af8-4acf-47e1-af64-c93cb361f273 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:07.256991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0709af8-4acf-47e1-af64-c93cb361f273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:07.274461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1307/2640] 10.222.0.59 () {58 vars in 967 bytes} [Fri Sep 16 08:13:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:13:07.290448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40e865-4cf0-4920-875a-813d894166ca None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:07.291911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bb40e865-4cf0-4920-875a-813d894166ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:07.294910 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f0709af8-4acf-47e1-af64-c93cb361f273 None None] MFA Rules not processed for user `2e3546dc78074fb68a12fc8a9d920be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:07.296224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f0709af8-4acf-47e1-af64-c93cb361f273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:07.334858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:07.376442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40e865-4cf0-4920-875a-813d894166ca None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:07.381656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1334/2641] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:07.385575 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0709af8-4acf-47e1-af64-c93cb361f273 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:07.439367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:07.449990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec046d2-3c6c-4b28-a15c-2d3286d17bbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:07.449990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec046d2-3c6c-4b28-a15c-2d3286d17bbc service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:07.449990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec046d2-3c6c-4b28-a15c-2d3286d17bbc service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:07.449990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec046d2-3c6c-4b28-a15c-2d3286d17bbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:07.467727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0709af8-4acf-47e1-af64-c93cb361f273 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:07.473232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fc443-6090-4728-b3ec-504518c0eed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:07.474030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fc443-6090-4728-b3ec-504518c0eed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:07.474499 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fc443-6090-4728-b3ec-504518c0eed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:07.475020 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fc443-6090-4728-b3ec-504518c0eed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:07.475815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fc443-6090-4728-b3ec-504518c0eed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:07.478459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fc443-6090-4728-b3ec-504518c0eed0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:07.519030 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec046d2-3c6c-4b28-a15c-2d3286d17bbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:07.520770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec046d2-3c6c-4b28-a15c-2d3286d17bbc service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:07.529713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1308/2642] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:13:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 16 08:13:07.575766 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1335/2643] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:13:07 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:13:07.576750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec046d2-3c6c-4b28-a15c-2d3286d17bbc service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:07.588361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec046d2-3c6c-4b28-a15c-2d3286d17bbc service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:07.649429 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ccfbc-4175-4c8b-af0b-36db7c7da9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:07.650039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ccfbc-4175-4c8b-af0b-36db7c7da9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:07.650321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ccfbc-4175-4c8b-af0b-36db7c7da9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:07.650600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ccfbc-4175-4c8b-af0b-36db7c7da9f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:07.651276 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ccfbc-4175-4c8b-af0b-36db7c7da9f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:07.653473 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ccfbc-4175-4c8b-af0b-36db7c7da9f4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:07.744317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1309/2644] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:13:07 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:13:07.749554 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fc443-6090-4728-b3ec-504518c0eed0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:07.763741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fc443-6090-4728-b3ec-504518c0eed0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:07.766999 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ebdcf-a0de-4643-832d-7431b958cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:07.767782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ebdcf-a0de-4643-832d-7431b958cef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:07.768937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ebdcf-a0de-4643-832d-7431b958cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:07.769324 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ebdcf-a0de-4643-832d-7431b958cef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:07.770130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ebdcf-a0de-4643-832d-7431b958cef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:07.771982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ebdcf-a0de-4643-832d-7431b958cef0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:07.785052 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1336/2645] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:07.789640 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ccfbc-4175-4c8b-af0b-36db7c7da9f4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:07.800136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ccfbc-4175-4c8b-af0b-36db7c7da9f4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:07.806258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c399d23-7a64-49ce-890f-1611e71cf8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:07.807239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c399d23-7a64-49ce-890f-1611e71cf8dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:07.807633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c399d23-7a64-49ce-890f-1611e71cf8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:07.807798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c399d23-7a64-49ce-890f-1611e71cf8dc admin admin] PATH_INFO: `/v3/projects/1fa0b2a494a14860be6328395f7c9072/users/f82b870260884171a87702a1b00b5f64/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:07.828226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c399d23-7a64-49ce-890f-1611e71cf8dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa0b2a494a14860be6328395f7c9072, user_id=f82b870260884171a87702a1b00b5f64, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:07.830964 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c399d23-7a64-49ce-890f-1611e71cf8dc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:07.922934 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1310/2646] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:13:07 2022] PUT /identity/v3/projects/1fa0b2a494a14860be6328395f7c9072/users/f82b870260884171a87702a1b00b5f64/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:07.923395 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ebdcf-a0de-4643-832d-7431b958cef0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:07.934266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ebdcf-a0de-4643-832d-7431b958cef0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:07.994089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188e1fe-ddee-49db-b28d-7a6a7ce4bc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:07.994627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188e1fe-ddee-49db-b28d-7a6a7ce4bc78 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:07.994942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188e1fe-ddee-49db-b28d-7a6a7ce4bc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:07.995221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188e1fe-ddee-49db-b28d-7a6a7ce4bc78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:07.995868 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188e1fe-ddee-49db-b28d-7a6a7ce4bc78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:07.997505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188e1fe-ddee-49db-b28d-7a6a7ce4bc78 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:08.007316 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1337/2647] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:08.012266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c399d23-7a64-49ce-890f-1611e71cf8dc admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:08.024672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c399d23-7a64-49ce-890f-1611e71cf8dc admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:08.029800 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30af82-f124-4d7f-ae51-a1c59500cb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:08.030535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30af82-f124-4d7f-ae51-a1c59500cb28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:08.030779 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30af82-f124-4d7f-ae51-a1c59500cb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:08.031072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30af82-f124-4d7f-ae51-a1c59500cb28 admin admin] PATH_INFO: `/v3/projects/1fa0b2a494a14860be6328395f7c9072/users/f82b870260884171a87702a1b00b5f64/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:08.033023 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30af82-f124-4d7f-ae51-a1c59500cb28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa0b2a494a14860be6328395f7c9072, user_id=f82b870260884171a87702a1b00b5f64, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:08.035762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30af82-f124-4d7f-ae51-a1c59500cb28 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:08.046586 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1311/2648] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:13:08 2022] PUT /identity/v3/projects/1fa0b2a494a14860be6328395f7c9072/users/f82b870260884171a87702a1b00b5f64/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:08.056783 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cab8b8-f4ce-4654-bc56-70c7903dd48e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:08.057253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cab8b8-f4ce-4654-bc56-70c7903dd48e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:08.057615 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cab8b8-f4ce-4654-bc56-70c7903dd48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:08.086909 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-60cab8b8-f4ce-4654-bc56-70c7903dd48e None None] MFA Rules not processed for user `f82b870260884171a87702a1b00b5f64`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:08.088116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-60cab8b8-f4ce-4654-bc56-70c7903dd48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:08.121722 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:08.162441 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1338/2649] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:08.172973 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30af82-f124-4d7f-ae51-a1c59500cb28 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:08.174751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0e30af82-f124-4d7f-ae51-a1c59500cb28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:08.240561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30af82-f124-4d7f-ae51-a1c59500cb28 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:08.244703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec194703-b9c1-4d5f-a875-7b2786c8c7a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:08.245226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec194703-b9c1-4d5f-a875-7b2786c8c7a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:08.245501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec194703-b9c1-4d5f-a875-7b2786c8c7a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:08.245770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec194703-b9c1-4d5f-a875-7b2786c8c7a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:08.256974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec194703-b9c1-4d5f-a875-7b2786c8c7a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:08.259060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec194703-b9c1-4d5f-a875-7b2786c8c7a2 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:08.263068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:08.269575 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1312/2650] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:13:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:13:08.411415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cab8b8-f4ce-4654-bc56-70c7903dd48e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:08.477466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cab8b8-f4ce-4654-bc56-70c7903dd48e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:08.482296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:08.483016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:08.483299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:08.483574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] PATH_INFO: `/v3/users/2e3546dc78074fb68a12fc8a9d920be1` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:08.484716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e3546dc78074fb68a12fc8a9d920be1)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:08.486664 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:08.635811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e3546dc78074fb68a12fc8a9d920be1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:08.649130 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e3546dc78074fb68a12fc8a9d920be1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:08.657678 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e3546dc78074fb68a12fc8a9d920be1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:08.729184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e3546dc78074fb68a12fc8a9d920be1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:08.729828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e3546dc78074fb68a12fc8a9d920be1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:08.739743 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1339/2651] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:13:08 2022] DELETE /identity/v3/users/2e3546dc78074fb68a12fc8a9d920be1 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:08.741381 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e645a-5a7b-4640-9b68-37474807d1ad None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:08.741856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e645a-5a7b-4640-9b68-37474807d1ad None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:08.742003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e645a-5a7b-4640-9b68-37474807d1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:08.761449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-199e645a-5a7b-4640-9b68-37474807d1ad None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:08.767323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-199e645a-5a7b-4640-9b68-37474807d1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:08.824212 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1313/2652] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:08.833993 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:08.835452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:08.890701 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:08.896696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586478c5-ef32-4cf1-8110-f57fb15bb284 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:08.900413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75cffd-78fd-4577-ab46-d503b2b642c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:08.901021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75cffd-78fd-4577-ab46-d503b2b642c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:08.901322 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75cffd-78fd-4577-ab46-d503b2b642c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:08.901590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75cffd-78fd-4577-ab46-d503b2b642c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:08.911544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75cffd-78fd-4577-ab46-d503b2b642c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:08.913248 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75cffd-78fd-4577-ab46-d503b2b642c2 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:08.926850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1340/2653] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:13:09.446062 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e645a-5a7b-4640-9b68-37474807d1ad None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:09.449335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-199e645a-5a7b-4640-9b68-37474807d1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:09.460481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e645a-5a7b-4640-9b68-37474807d1ad None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:09.466467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:09.467234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:09.467569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:09.467936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] PATH_INFO: `/v3/projects/940c02c35da34b9081d98ae0aeddfbdd` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:09.469102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=940c02c35da34b9081d98ae0aeddfbdd)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:09.471144 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:09.553837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940c02c35da34b9081d98ae0aeddfbdd', 'request_id': 'req-918ad9d9-394e-4d59-976d-8fe7501effa3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:09.561029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940c02c35da34b9081d98ae0aeddfbdd', 'request_id': 'req-918ad9d9-394e-4d59-976d-8fe7501effa3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:09.566776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940c02c35da34b9081d98ae0aeddfbdd', 'request_id': 'req-918ad9d9-394e-4d59-976d-8fe7501effa3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:09.620932 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] The token cache is being invalidate because project 940c02c35da34b9081d98ae0aeddfbdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:13:09.621224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:09.621878 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:09.623388 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1314/2654] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:13:09 2022] DELETE /identity/v3/projects/940c02c35da34b9081d98ae0aeddfbdd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:09.627338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75cffd-78fd-4577-ab46-d503b2b642c2 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:09.638400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75cffd-78fd-4577-ab46-d503b2b642c2 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:09.696393 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c04f5e-4048-4da2-a2a6-0abbf34d551b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:09.696940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c04f5e-4048-4da2-a2a6-0abbf34d551b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:09.697218 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c04f5e-4048-4da2-a2a6-0abbf34d551b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:09.697488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c04f5e-4048-4da2-a2a6-0abbf34d551b admin admin] PATH_INFO: `/v3/users/f82b870260884171a87702a1b00b5f64` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:09.698614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c04f5e-4048-4da2-a2a6-0abbf34d551b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f82b870260884171a87702a1b00b5f64)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:09.700546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c04f5e-4048-4da2-a2a6-0abbf34d551b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:09.830231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e5c04f5e-4048-4da2-a2a6-0abbf34d551b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f82b870260884171a87702a1b00b5f64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:09.842454 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e5c04f5e-4048-4da2-a2a6-0abbf34d551b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f82b870260884171a87702a1b00b5f64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:09.850213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e5c04f5e-4048-4da2-a2a6-0abbf34d551b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f82b870260884171a87702a1b00b5f64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:09.907112 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e5c04f5e-4048-4da2-a2a6-0abbf34d551b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f82b870260884171a87702a1b00b5f64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:09.907737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e5c04f5e-4048-4da2-a2a6-0abbf34d551b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f82b870260884171a87702a1b00b5f64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:09.914097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1341/2655] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:13:09 2022] DELETE /identity/v3/users/f82b870260884171a87702a1b00b5f64 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:10.462225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:10.463840 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:10.479526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ad9d9-394e-4d59-976d-8fe7501effa3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:10.540621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:10.541195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:10.541482 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:10.541835 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] PATH_INFO: `/v3/projects/1fa0b2a494a14860be6328395f7c9072` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:10.542907 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa0b2a494a14860be6328395f7c9072)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:10.544821 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:10.569342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:10.656905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa0b2a494a14860be6328395f7c9072', 'request_id': 'req-25fd805c-ba39-48ba-940f-eeb74db11c79'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:10.664826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa0b2a494a14860be6328395f7c9072', 'request_id': 'req-25fd805c-ba39-48ba-940f-eeb74db11c79'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:10.670621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa0b2a494a14860be6328395f7c9072', 'request_id': 'req-25fd805c-ba39-48ba-940f-eeb74db11c79'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:10.769553 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] The token cache is being invalidate because project 1fa0b2a494a14860be6328395f7c9072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:13:10.769864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:10.770512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-25fd805c-ba39-48ba-940f-eeb74db11c79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:10.772029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1315/2656] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:13:10 2022] DELETE /identity/v3/projects/1fa0b2a494a14860be6328395f7c9072 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:10.777330 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaec48-80a0-4e10-906d-8970395a2116 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:10.777827 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaec48-80a0-4e10-906d-8970395a2116 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:10.777932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaec48-80a0-4e10-906d-8970395a2116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:10.798781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-52eaec48-80a0-4e10-906d-8970395a2116 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:10.800289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-52eaec48-80a0-4e10-906d-8970395a2116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:10.863707 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1342/2657] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:10.871978 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5353e-97b1-40ff-a202-8fc0671fda1a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:10.872269 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5353e-97b1-40ff-a202-8fc0671fda1a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:10.872549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5353e-97b1-40ff-a202-8fc0671fda1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:10.893893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5ae5353e-97b1-40ff-a202-8fc0671fda1a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:10.895155 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5ae5353e-97b1-40ff-a202-8fc0671fda1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:10.904009 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1316/2658] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:13:10.908423 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaec48-80a0-4e10-906d-8970395a2116 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:10.918490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaec48-80a0-4e10-906d-8970395a2116 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:10.923985 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8c6f4-a50a-47bd-b767-60bdba76e667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:10.924773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8c6f4-a50a-47bd-b767-60bdba76e667 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:10.925062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8c6f4-a50a-47bd-b767-60bdba76e667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:10.925341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8c6f4-a50a-47bd-b767-60bdba76e667 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:10.926021 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8c6f4-a50a-47bd-b767-60bdba76e667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:13:10.926349 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8c6f4-a50a-47bd-b767-60bdba76e667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:10.928076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8c6f4-a50a-47bd-b767-60bdba76e667 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:10.930982 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:10.940784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1343/2659] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:13:10.948735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5353e-97b1-40ff-a202-8fc0671fda1a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:10.958246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5353e-97b1-40ff-a202-8fc0671fda1a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:10.965044 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b7f44-2fb2-47a3-b7ec-88ca0ba690a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:10.965708 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b7f44-2fb2-47a3-b7ec-88ca0ba690a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:10.965993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b7f44-2fb2-47a3-b7ec-88ca0ba690a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:10.966264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b7f44-2fb2-47a3-b7ec-88ca0ba690a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:10.966933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b7f44-2fb2-47a3-b7ec-88ca0ba690a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:10.968921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b7f44-2fb2-47a3-b7ec-88ca0ba690a2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:11.073876 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1317/2660] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:13:10 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:13:11.075553 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8c6f4-a50a-47bd-b767-60bdba76e667 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:11.085350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8c6f4-a50a-47bd-b767-60bdba76e667 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:11.147642 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf122c1-9ced-460d-a5fc-fd129e44c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:11.148793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf122c1-9ced-460d-a5fc-fd129e44c9ae admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:11.149070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf122c1-9ced-460d-a5fc-fd129e44c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:11.149426 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf122c1-9ced-460d-a5fc-fd129e44c9ae admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:11.150078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf122c1-9ced-460d-a5fc-fd129e44c9ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:11.152310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf122c1-9ced-460d-a5fc-fd129e44c9ae admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:11.262483 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1344/2661] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:13:11 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:13:11.266930 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b7f44-2fb2-47a3-b7ec-88ca0ba690a2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:11.279061 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b7f44-2fb2-47a3-b7ec-88ca0ba690a2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:11.283637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0981e68-6c52-43be-bc96-191b45b7ec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:11.284249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0981e68-6c52-43be-bc96-191b45b7ec7d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:11.287597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0981e68-6c52-43be-bc96-191b45b7ec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:11.287597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0981e68-6c52-43be-bc96-191b45b7ec7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:11.287597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0981e68-6c52-43be-bc96-191b45b7ec7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:11.287597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0981e68-6c52-43be-bc96-191b45b7ec7d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:11.297845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1318/2662] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:11.301972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf122c1-9ced-460d-a5fc-fd129e44c9ae admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:11.313490 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf122c1-9ced-460d-a5fc-fd129e44c9ae admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:11.321232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0a512-9991-449a-a245-266230649c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:11.321856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a512-9991-449a-a245-266230649c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:11.322266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a512-9991-449a-a245-266230649c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:11.322587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a512-9991-449a-a245-266230649c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:11.323159 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0a512-9991-449a-a245-266230649c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:11.324758 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0a512-9991-449a-a245-266230649c09 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:11.338783 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1345/2663] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:11.340105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0981e68-6c52-43be-bc96-191b45b7ec7d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:11.350585 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0981e68-6c52-43be-bc96-191b45b7ec7d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:11.355515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173093df-7874-4225-af18-188f721cd8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:11.356169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173093df-7874-4225-af18-188f721cd8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:11.356447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173093df-7874-4225-af18-188f721cd8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:11.356748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173093df-7874-4225-af18-188f721cd8f7 admin admin] PATH_INFO: `/v3/projects/2f61daeba10243e58eedffcd5c1f59e5/users/f6e004842f854e0d89a639c3f33e79ab/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:11.372584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173093df-7874-4225-af18-188f721cd8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f61daeba10243e58eedffcd5c1f59e5, user_id=f6e004842f854e0d89a639c3f33e79ab, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:11.375300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173093df-7874-4225-af18-188f721cd8f7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:11.454815 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1319/2664] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:13:11 2022] PUT /identity/v3/projects/2f61daeba10243e58eedffcd5c1f59e5/users/f6e004842f854e0d89a639c3f33e79ab/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:11.457532 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbfe47-ee25-44c1-a0d4-89ad084e1385 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:11.457825 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbfe47-ee25-44c1-a0d4-89ad084e1385 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:11.458124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbfe47-ee25-44c1-a0d4-89ad084e1385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:11.488448 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a0dbfe47-ee25-44c1-a0d4-89ad084e1385 None None] MFA Rules not processed for user `f6e004842f854e0d89a639c3f33e79ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:11.489717 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a0dbfe47-ee25-44c1-a0d4-89ad084e1385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:11.574784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1346/2665] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:11.586049 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173093df-7874-4225-af18-188f721cd8f7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:11.589573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-173093df-7874-4225-af18-188f721cd8f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:11.649261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173093df-7874-4225-af18-188f721cd8f7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:11.653293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93eef-f46a-445c-bb09-6b83be12feb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:11.653899 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93eef-f46a-445c-bb09-6b83be12feb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:11.654176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93eef-f46a-445c-bb09-6b83be12feb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:11.654447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93eef-f46a-445c-bb09-6b83be12feb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:11.662532 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad93eef-f46a-445c-bb09-6b83be12feb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:11.664516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad93eef-f46a-445c-bb09-6b83be12feb8 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:11.673314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1320/2666] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:13:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:13:17.631484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbfe47-ee25-44c1-a0d4-89ad084e1385 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:17.633446 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a0dbfe47-ee25-44c1-a0d4-89ad084e1385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:17.645509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbfe47-ee25-44c1-a0d4-89ad084e1385 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:17.649923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ed2f8-99c1-4795-8022-f2dce7d6c1e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:17.650529 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ed2f8-99c1-4795-8022-f2dce7d6c1e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:17.650974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ed2f8-99c1-4795-8022-f2dce7d6c1e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:17.651252 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ed2f8-99c1-4795-8022-f2dce7d6c1e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:17.653194 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5e3ed2f8-99c1-4795-8022-f2dce7d6c1e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:17.711876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ed2f8-99c1-4795-8022-f2dce7d6c1e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:17.713336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ed2f8-99c1-4795-8022-f2dce7d6c1e6 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:17.722512 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1347/2667] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:13:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:13:21.669470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=68,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:24.589867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93eef-f46a-445c-bb09-6b83be12feb8 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:24.653365 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93eef-f46a-445c-bb09-6b83be12feb8 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:24.657916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:24.658608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:24.658912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:24.659182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] PATH_INFO: `/v3/users/f6e004842f854e0d89a639c3f33e79ab` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:24.660281 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e004842f854e0d89a639c3f33e79ab)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:24.662206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:24.828521 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e004842f854e0d89a639c3f33e79ab'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:24.842034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e004842f854e0d89a639c3f33e79ab'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:24.849091 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e004842f854e0d89a639c3f33e79ab'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:24.906458 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e004842f854e0d89a639c3f33e79ab'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:24.907092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e004842f854e0d89a639c3f33e79ab'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:24.913330 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1321/2668] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:13:24 2022] DELETE /identity/v3/users/f6e004842f854e0d89a639c3f33e79ab => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:24.918655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97bcce-4b9d-4596-8418-050bd911aaa4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:24.919149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97bcce-4b9d-4596-8418-050bd911aaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:24.919249 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97bcce-4b9d-4596-8418-050bd911aaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:24.926455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:24.940138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7b97bcce-4b9d-4596-8418-050bd911aaa4 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:24.941363 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7b97bcce-4b9d-4596-8418-050bd911aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:24.999515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1348/2669] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:13:25.009704 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:25.011729 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:25.090162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d632233-ceae-4b93-b7e9-fd278d24ce14 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:25.093917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9be06-2200-449a-877b-d568ccab7b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:25.094536 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9be06-2200-449a-877b-d568ccab7b50 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:25.094894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9be06-2200-449a-877b-d568ccab7b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:25.095169 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9be06-2200-449a-877b-d568ccab7b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:25.105905 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9be06-2200-449a-877b-d568ccab7b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:25.107531 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9be06-2200-449a-877b-d568ccab7b50 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:25.118260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1322/2670] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:13:25.549472 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97bcce-4b9d-4596-8418-050bd911aaa4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:25.551981 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7b97bcce-4b9d-4596-8418-050bd911aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:25.571315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97bcce-4b9d-4596-8418-050bd911aaa4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:25.582397 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:25.583212 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:25.583622 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:25.583988 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] PATH_INFO: `/v3/projects/2f61daeba10243e58eedffcd5c1f59e5` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:25.585203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f61daeba10243e58eedffcd5c1f59e5)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:25.587856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:25.738304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f61daeba10243e58eedffcd5c1f59e5', 'request_id': 'req-be6ae65c-c5d4-41d7-87e7-a342b617c852'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:25.744857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f61daeba10243e58eedffcd5c1f59e5', 'request_id': 'req-be6ae65c-c5d4-41d7-87e7-a342b617c852'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:25.750455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f61daeba10243e58eedffcd5c1f59e5', 'request_id': 'req-be6ae65c-c5d4-41d7-87e7-a342b617c852'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:25.837894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] The token cache is being invalidate because project 2f61daeba10243e58eedffcd5c1f59e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:13:25.838192 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:25.838858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be6ae65c-c5d4-41d7-87e7-a342b617c852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:25.840338 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1349/2671] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:13:25 2022] DELETE /identity/v3/projects/2f61daeba10243e58eedffcd5c1f59e5 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:25.845183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ea2e4-d0e6-4791-888b-fd8f986efff2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:25.845657 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ea2e4-d0e6-4791-888b-fd8f986efff2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:25.845777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ea2e4-d0e6-4791-888b-fd8f986efff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:25.866360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fd3ea2e4-d0e6-4791-888b-fd8f986efff2 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:25.867937 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fd3ea2e4-d0e6-4791-888b-fd8f986efff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:25.886143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:25.927340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1323/2672] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:25.997594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53219f-5e80-4ec6-a6a2-ba1627f9b658 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:25.997594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53219f-5e80-4ec6-a6a2-ba1627f9b658 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:25.997594 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53219f-5e80-4ec6-a6a2-ba1627f9b658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:26.022726 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6a53219f-5e80-4ec6-a6a2-ba1627f9b658 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:26.024509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6a53219f-5e80-4ec6-a6a2-ba1627f9b658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:26.036006 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1350/2673] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:26.037233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ea2e4-d0e6-4791-888b-fd8f986efff2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:26.047958 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ea2e4-d0e6-4791-888b-fd8f986efff2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:26.052595 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc3fa0-cf21-4339-a0f9-9d75b670ddd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:26.053274 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc3fa0-cf21-4339-a0f9-9d75b670ddd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:26.053682 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc3fa0-cf21-4339-a0f9-9d75b670ddd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:26.053845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc3fa0-cf21-4339-a0f9-9d75b670ddd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:26.054486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc3fa0-cf21-4339-a0f9-9d75b670ddd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:13:26.054858 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc3fa0-cf21-4339-a0f9-9d75b670ddd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:26.056400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc3fa0-cf21-4339-a0f9-9d75b670ddd2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:26.068901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1324/2674] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:13:26.073943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53219f-5e80-4ec6-a6a2-ba1627f9b658 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:26.083816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53219f-5e80-4ec6-a6a2-ba1627f9b658 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:26.090271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1ffbc-9323-4c56-8163-7ebd59c2674b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:26.090271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1ffbc-9323-4c56-8163-7ebd59c2674b admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:26.090271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1ffbc-9323-4c56-8163-7ebd59c2674b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:26.090271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1ffbc-9323-4c56-8163-7ebd59c2674b admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:26.090583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1ffbc-9323-4c56-8163-7ebd59c2674b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:26.092177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1ffbc-9323-4c56-8163-7ebd59c2674b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:26.165584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1351/2675] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:13:26 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:13:26.169892 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc3fa0-cf21-4339-a0f9-9d75b670ddd2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:26.181056 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc3fa0-cf21-4339-a0f9-9d75b670ddd2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:26.238492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19770d74-b0e7-4d0d-9c64-f72788250875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:26.239138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19770d74-b0e7-4d0d-9c64-f72788250875 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:26.239421 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19770d74-b0e7-4d0d-9c64-f72788250875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:26.239700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19770d74-b0e7-4d0d-9c64-f72788250875 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:26.240331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19770d74-b0e7-4d0d-9c64-f72788250875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:26.242821 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19770d74-b0e7-4d0d-9c64-f72788250875 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:26.364071 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1325/2676] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:13:26 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:13:26.368462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1ffbc-9323-4c56-8163-7ebd59c2674b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:26.378882 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1ffbc-9323-4c56-8163-7ebd59c2674b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:26.383715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b11016-83b0-4287-9cc8-aacf8bda0ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:26.384313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b11016-83b0-4287-9cc8-aacf8bda0ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:26.384595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b11016-83b0-4287-9cc8-aacf8bda0ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:26.384927 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b11016-83b0-4287-9cc8-aacf8bda0ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:26.385544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b11016-83b0-4287-9cc8-aacf8bda0ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:26.387171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b11016-83b0-4287-9cc8-aacf8bda0ab3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:26.397183 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1352/2677] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:26.401512 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19770d74-b0e7-4d0d-9c64-f72788250875 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:26.411752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19770d74-b0e7-4d0d-9c64-f72788250875 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:26.417335 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af32ce-b421-4b98-9664-4833c174eeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:26.417906 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af32ce-b421-4b98-9664-4833c174eeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:26.418187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af32ce-b421-4b98-9664-4833c174eeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:26.418461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af32ce-b421-4b98-9664-4833c174eeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:26.419223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af32ce-b421-4b98-9664-4833c174eeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:26.420794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af32ce-b421-4b98-9664-4833c174eeb3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:26.431336 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1326/2678] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:26.435322 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b11016-83b0-4287-9cc8-aacf8bda0ab3 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:26.445494 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b11016-83b0-4287-9cc8-aacf8bda0ab3 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:26.451782 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e737c98-5464-418f-8b11-ec3a25fff82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:26.452422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e737c98-5464-418f-8b11-ec3a25fff82b admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:26.452698 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e737c98-5464-418f-8b11-ec3a25fff82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:26.452964 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e737c98-5464-418f-8b11-ec3a25fff82b admin admin] PATH_INFO: `/v3/projects/6870075020534e76a41acb1353badb39/users/c16e2c0456d341e7bfdbf4d10b3e223e/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:26.467436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e737c98-5464-418f-8b11-ec3a25fff82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6870075020534e76a41acb1353badb39, user_id=c16e2c0456d341e7bfdbf4d10b3e223e, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:26.470051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e737c98-5464-418f-8b11-ec3a25fff82b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:26.522528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1353/2679] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:13:26 2022] PUT /identity/v3/projects/6870075020534e76a41acb1353badb39/users/c16e2c0456d341e7bfdbf4d10b3e223e/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:13:26.533095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb689e6-47a1-4ea0-a44e-8e5c7fadadf4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:26.533360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb689e6-47a1-4ea0-a44e-8e5c7fadadf4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:26.533697 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb689e6-47a1-4ea0-a44e-8e5c7fadadf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:26.562287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fcb689e6-47a1-4ea0-a44e-8e5c7fadadf4 None None] MFA Rules not processed for user `c16e2c0456d341e7bfdbf4d10b3e223e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:26.563539 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fcb689e6-47a1-4ea0-a44e-8e5c7fadadf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:26.629246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1327/2680] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:26.640937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e737c98-5464-418f-8b11-ec3a25fff82b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:26.642696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9e737c98-5464-418f-8b11-ec3a25fff82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:26.669231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:26.701656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e737c98-5464-418f-8b11-ec3a25fff82b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:26.705310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee4aff-eb86-4eaf-8c9f-0436838107bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:26.705915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4aff-eb86-4eaf-8c9f-0436838107bd service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:26.706217 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4aff-eb86-4eaf-8c9f-0436838107bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:26.706475 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4aff-eb86-4eaf-8c9f-0436838107bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:26.716654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee4aff-eb86-4eaf-8c9f-0436838107bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:26.718119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee4aff-eb86-4eaf-8c9f-0436838107bd service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:26.728563 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1354/2681] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:13:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:13:26.755902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb689e6-47a1-4ea0-a44e-8e5c7fadadf4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:26.776593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:26.822761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb689e6-47a1-4ea0-a44e-8e5c7fadadf4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:26.829807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:26.835098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:26.835098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:26.835098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] PATH_INFO: `/v3/users/c16e2c0456d341e7bfdbf4d10b3e223e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:26.835098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c16e2c0456d341e7bfdbf4d10b3e223e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:26.835098 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:27.007410 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c16e2c0456d341e7bfdbf4d10b3e223e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:27.022551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c16e2c0456d341e7bfdbf4d10b3e223e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:27.032024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c16e2c0456d341e7bfdbf4d10b3e223e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:27.095193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c16e2c0456d341e7bfdbf4d10b3e223e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:27.096078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c16e2c0456d341e7bfdbf4d10b3e223e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:27.104312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1328/2682] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:13:26 2022] DELETE /identity/v3/users/c16e2c0456d341e7bfdbf4d10b3e223e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:27.108952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa47a2a-9be8-4f92-b0f6-0aec73baa92b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:27.109437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa47a2a-9be8-4f92-b0f6-0aec73baa92b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:27.109708 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa47a2a-9be8-4f92-b0f6-0aec73baa92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:27.133903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cfa47a2a-9be8-4f92-b0f6-0aec73baa92b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:27.135175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cfa47a2a-9be8-4f92-b0f6-0aec73baa92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:27.197445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1355/2683] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:27.207153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:27.212289 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:27.267467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12109b29-ee6c-4d97-994c-059cb97a6056 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:27.271215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46465725-4118-429c-a268-98763226f4d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:27.271823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46465725-4118-429c-a268-98763226f4d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:27.272112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46465725-4118-429c-a268-98763226f4d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:27.272561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46465725-4118-429c-a268-98763226f4d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:27.283072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46465725-4118-429c-a268-98763226f4d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:27.284610 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46465725-4118-429c-a268-98763226f4d5 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:27.294812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1329/2684] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:13:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:13:27.783723 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa47a2a-9be8-4f92-b0f6-0aec73baa92b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:27.786712 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cfa47a2a-9be8-4f92-b0f6-0aec73baa92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:27.802939 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa47a2a-9be8-4f92-b0f6-0aec73baa92b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:27.813563 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:27.814234 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:27.814547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:27.815026 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] PATH_INFO: `/v3/projects/6870075020534e76a41acb1353badb39` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:27.816232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=6870075020534e76a41acb1353badb39)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:27.818151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:27.958790 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6870075020534e76a41acb1353badb39', 'request_id': 'req-7e275f9a-fb94-4dcc-8b46-0775e97413ab'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:27.965926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6870075020534e76a41acb1353badb39', 'request_id': 'req-7e275f9a-fb94-4dcc-8b46-0775e97413ab'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:27.972892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6870075020534e76a41acb1353badb39', 'request_id': 'req-7e275f9a-fb94-4dcc-8b46-0775e97413ab'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:28.055528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] The token cache is being invalidate because project 6870075020534e76a41acb1353badb39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:13:28.056057 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:28.056902 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7e275f9a-fb94-4dcc-8b46-0775e97413ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:28.058604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1356/2685] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:13:27 2022] DELETE /identity/v3/projects/6870075020534e76a41acb1353badb39 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:28.384863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdff28b-c75f-4dcc-a65d-6cfa699d0faa None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:28.385985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdff28b-c75f-4dcc-a65d-6cfa699d0faa None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:28.386567 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdff28b-c75f-4dcc-a65d-6cfa699d0faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:28.410366 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1cdff28b-c75f-4dcc-a65d-6cfa699d0faa None None] MFA Rules not processed for user `afbb1d9c9bb34fbd8bfc78710c86d1f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:28.431477 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1cdff28b-c75f-4dcc-a65d-6cfa699d0faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:28.449511 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:28.471684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1330/2686] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:33.827685 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568979b-d1a7-46a3-9ec3-64a97cb30808 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:33.828319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568979b-d1a7-46a3-9ec3-64a97cb30808 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:33.828319 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568979b-d1a7-46a3-9ec3-64a97cb30808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:33.847525 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7568979b-d1a7-46a3-9ec3-64a97cb30808 None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:33.848718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7568979b-d1a7-46a3-9ec3-64a97cb30808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:33.894380 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:33.913024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1357/2687] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:13:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:33.936804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdff28b-c75f-4dcc-a65d-6cfa699d0faa None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:33.939352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1cdff28b-c75f-4dcc-a65d-6cfa699d0faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:34.012660 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdff28b-c75f-4dcc-a65d-6cfa699d0faa None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:34.017802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691383d1-4cbd-4c66-a8d6-7ff2fe4492cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:34.018408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691383d1-4cbd-4c66-a8d6-7ff2fe4492cf service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:34.018714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691383d1-4cbd-4c66-a8d6-7ff2fe4492cf service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:34.018985 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691383d1-4cbd-4c66-a8d6-7ff2fe4492cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:34.027135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691383d1-4cbd-4c66-a8d6-7ff2fe4492cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:34.029005 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691383d1-4cbd-4c66-a8d6-7ff2fe4492cf service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:34.036872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1331/2688] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:13:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:13:43.969150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=6 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:44.033832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=39 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:51.885455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e0b8-a7a7-4542-9edc-15e6c8533908 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:51.885455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e0b8-a7a7-4542-9edc-15e6c8533908 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:51.885455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e0b8-a7a7-4542-9edc-15e6c8533908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:51.909382 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a434e0b8-a7a7-4542-9edc-15e6c8533908 None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:51.910659 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a434e0b8-a7a7-4542-9edc-15e6c8533908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:51.919817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1358/2689] 10.222.0.59 () {58 vars in 967 bytes} [Fri Sep 16 08:13:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:51.932401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691383d1-4cbd-4c66-a8d6-7ff2fe4492cf service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:51.944801 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691383d1-4cbd-4c66-a8d6-7ff2fe4492cf service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:51.950928 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e764c-aeec-45eb-93d2-f66c0070f0de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:51.956534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e764c-aeec-45eb-93d2-f66c0070f0de service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:51.956534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e764c-aeec-45eb-93d2-f66c0070f0de service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:51.956534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e764c-aeec-45eb-93d2-f66c0070f0de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:51.961663 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e764c-aeec-45eb-93d2-f66c0070f0de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:51.963550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e764c-aeec-45eb-93d2-f66c0070f0de service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:51.972886 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1332/2690] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:13:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:13:54.924336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434e0b8-a7a7-4542-9edc-15e6c8533908 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:54.925778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a434e0b8-a7a7-4542-9edc-15e6c8533908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:55.006843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434e0b8-a7a7-4542-9edc-15e6c8533908 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:55.011314 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbd421-5f7a-4902-9642-a90d61f0e61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:55.016603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbd421-5f7a-4902-9642-a90d61f0e61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:55.016603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbd421-5f7a-4902-9642-a90d61f0e61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:55.016603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbd421-5f7a-4902-9642-a90d61f0e61b admin admin] PATH_INFO: `/v3/users/e9cd1b533aa8456d95e1bdbeac678b9f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:55.016603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbd421-5f7a-4902-9642-a90d61f0e61b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9cd1b533aa8456d95e1bdbeac678b9f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:55.016603 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbd421-5f7a-4902-9642-a90d61f0e61b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:55.164939 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-26dbd421-5f7a-4902-9642-a90d61f0e61b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9cd1b533aa8456d95e1bdbeac678b9f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:55.177633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-26dbd421-5f7a-4902-9642-a90d61f0e61b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9cd1b533aa8456d95e1bdbeac678b9f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:55.184754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-26dbd421-5f7a-4902-9642-a90d61f0e61b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9cd1b533aa8456d95e1bdbeac678b9f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:55.248733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-26dbd421-5f7a-4902-9642-a90d61f0e61b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9cd1b533aa8456d95e1bdbeac678b9f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:55.249355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-26dbd421-5f7a-4902-9642-a90d61f0e61b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9cd1b533aa8456d95e1bdbeac678b9f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:55.255764 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1359/2691] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:13:54 2022] DELETE /identity/v3/users/e9cd1b533aa8456d95e1bdbeac678b9f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:55.513883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e764c-aeec-45eb-93d2-f66c0070f0de service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:55.526156 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e764c-aeec-45eb-93d2-f66c0070f0de service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:55.586099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:55.586903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:55.587393 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:55.587733 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] PATH_INFO: `/v3/projects/9d32f40c2bd64a61aadd27f82d11191a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:55.588864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d32f40c2bd64a61aadd27f82d11191a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:55.590935 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:55.701782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d32f40c2bd64a61aadd27f82d11191a', 'request_id': 'req-4669336e-d757-4510-841e-1950aaeda858'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:55.708342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d32f40c2bd64a61aadd27f82d11191a', 'request_id': 'req-4669336e-d757-4510-841e-1950aaeda858'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:55.714077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d32f40c2bd64a61aadd27f82d11191a', 'request_id': 'req-4669336e-d757-4510-841e-1950aaeda858'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:55.780115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] The token cache is being invalidate because project 9d32f40c2bd64a61aadd27f82d11191a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:13:55.780409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:55.781037 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4669336e-d757-4510-841e-1950aaeda858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:13:55.782608 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1333/2692] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:13:55 2022] DELETE /identity/v3/projects/9d32f40c2bd64a61aadd27f82d11191a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:55.788547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc3f75-bda3-4886-836e-3c81889871b9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:55.788841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc3f75-bda3-4886-836e-3c81889871b9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:55.789118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc3f75-bda3-4886-836e-3c81889871b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:55.809785 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-19bc3f75-bda3-4886-836e-3c81889871b9 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:55.811126 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-19bc3f75-bda3-4886-836e-3c81889871b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:55.868567 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1360/2693] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:55.877160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a366be8-7a2f-4f33-9684-acfb97dbc60d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:55.877471 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a366be8-7a2f-4f33-9684-acfb97dbc60d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:55.877746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a366be8-7a2f-4f33-9684-acfb97dbc60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:55.897474 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9a366be8-7a2f-4f33-9684-acfb97dbc60d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:55.898700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9a366be8-7a2f-4f33-9684-acfb97dbc60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:55.906743 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1334/2694] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:55.910936 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc3f75-bda3-4886-836e-3c81889871b9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:55.920972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc3f75-bda3-4886-836e-3c81889871b9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:55.925502 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab9203-e271-49a2-85d7-606b361665dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:55.926158 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab9203-e271-49a2-85d7-606b361665dd admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:55.926354 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab9203-e271-49a2-85d7-606b361665dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:55.926626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab9203-e271-49a2-85d7-606b361665dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:55.927306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab9203-e271-49a2-85d7-606b361665dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:13:55.927608 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab9203-e271-49a2-85d7-606b361665dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:55.929190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab9203-e271-49a2-85d7-606b361665dd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:55.938296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:55.941438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1361/2695] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:13:55.945805 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a366be8-7a2f-4f33-9684-acfb97dbc60d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:55.954977 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a366be8-7a2f-4f33-9684-acfb97dbc60d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:55.959597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71fb2c-e652-443b-8fad-8d231808b472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:55.960128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71fb2c-e652-443b-8fad-8d231808b472 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:55.960407 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71fb2c-e652-443b-8fad-8d231808b472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:55.960679 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71fb2c-e652-443b-8fad-8d231808b472 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:55.961313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71fb2c-e652-443b-8fad-8d231808b472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:55.963358 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71fb2c-e652-443b-8fad-8d231808b472 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:56.062187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1335/2696] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:13:55 2022] POST /identity/v3/projects => generated 389 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:13:56.066570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab9203-e271-49a2-85d7-606b361665dd admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:56.077453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab9203-e271-49a2-85d7-606b361665dd admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:56.134970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb4551-6433-4cc8-8670-7e037bf4731d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:56.135565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb4551-6433-4cc8-8670-7e037bf4731d admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:56.135843 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb4551-6433-4cc8-8670-7e037bf4731d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:56.136115 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb4551-6433-4cc8-8670-7e037bf4731d admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:56.136746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb4551-6433-4cc8-8670-7e037bf4731d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:56.138657 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb4551-6433-4cc8-8670-7e037bf4731d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:56.220581 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1362/2697] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:13:56 2022] POST /identity/v3/users => generated 295 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:13:56.224646 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71fb2c-e652-443b-8fad-8d231808b472 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:56.235523 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71fb2c-e652-443b-8fad-8d231808b472 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:56.240271 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592eee3e-f236-4637-80bd-6dcc5c514533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:56.240861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eee3e-f236-4637-80bd-6dcc5c514533 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:56.241138 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eee3e-f236-4637-80bd-6dcc5c514533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:56.241410 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eee3e-f236-4637-80bd-6dcc5c514533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:56.242051 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592eee3e-f236-4637-80bd-6dcc5c514533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:56.243667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592eee3e-f236-4637-80bd-6dcc5c514533 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:56.253493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1336/2698] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:56.257518 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb4551-6433-4cc8-8670-7e037bf4731d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:56.267853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb4551-6433-4cc8-8670-7e037bf4731d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:56.272761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f925a-ec00-439c-a84a-bca7677e0dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:56.273350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f925a-ec00-439c-a84a-bca7677e0dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:56.273652 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f925a-ec00-439c-a84a-bca7677e0dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:56.273923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f925a-ec00-439c-a84a-bca7677e0dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:56.274576 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f925a-ec00-439c-a84a-bca7677e0dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:56.276325 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f925a-ec00-439c-a84a-bca7677e0dbf admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:56.287604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1363/2699] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:13:56.291634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592eee3e-f236-4637-80bd-6dcc5c514533 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:56.302296 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592eee3e-f236-4637-80bd-6dcc5c514533 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:56.307101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe2782-2652-4d8e-a489-4ca158379f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:56.307755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe2782-2652-4d8e-a489-4ca158379f7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:56.308038 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe2782-2652-4d8e-a489-4ca158379f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:56.308311 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe2782-2652-4d8e-a489-4ca158379f7c admin admin] PATH_INFO: `/v3/projects/330421f977444ac9ab30f57e049464a5/users/8b333b03b4ab4ef59c492bdd6aa1ae44/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:56.322669 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe2782-2652-4d8e-a489-4ca158379f7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=330421f977444ac9ab30f57e049464a5, user_id=8b333b03b4ab4ef59c492bdd6aa1ae44, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:56.325470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe2782-2652-4d8e-a489-4ca158379f7c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:56.414823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1337/2700] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:13:56 2022] PUT /identity/v3/projects/330421f977444ac9ab30f57e049464a5/users/8b333b03b4ab4ef59c492bdd6aa1ae44/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:13:56.428303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9c786-1e22-4102-8442-d476cf5941c3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:56.428593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9c786-1e22-4102-8442-d476cf5941c3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:56.428875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9c786-1e22-4102-8442-d476cf5941c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:56.456801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-afd9c786-1e22-4102-8442-d476cf5941c3 None None] MFA Rules not processed for user `8b333b03b4ab4ef59c492bdd6aa1ae44`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:13:56.458005 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-afd9c786-1e22-4102-8442-d476cf5941c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:56.531670 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1364/2701] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:13:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:13:56.539319 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe2782-2652-4d8e-a489-4ca158379f7c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:13:56.541104 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a5fe2782-2652-4d8e-a489-4ca158379f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:13:56.556666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:13:56.606481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe2782-2652-4d8e-a489-4ca158379f7c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:13:56.611605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae55e41-185a-4a2e-8697-8b1d54667b4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:13:56.612021 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae55e41-185a-4a2e-8697-8b1d54667b4f service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:13:56.612413 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae55e41-185a-4a2e-8697-8b1d54667b4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:13:56.612893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae55e41-185a-4a2e-8697-8b1d54667b4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:13:56.621931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae55e41-185a-4a2e-8697-8b1d54667b4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:13:56.623437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae55e41-185a-4a2e-8697-8b1d54667b4f service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:13:56.632760 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1338/2702] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:13:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:14:02.519072 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9c786-1e22-4102-8442-d476cf5941c3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:02.520540 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-afd9c786-1e22-4102-8442-d476cf5941c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:02.591460 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:02.616095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9c786-1e22-4102-8442-d476cf5941c3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:02.621180 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a2110-8258-437a-bccc-afc1b244d09e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:02.622111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a2110-8258-437a-bccc-afc1b244d09e service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:02.622111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a2110-8258-437a-bccc-afc1b244d09e service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:02.622541 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a2110-8258-437a-bccc-afc1b244d09e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:02.624950 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a08a2110-8258-437a-bccc-afc1b244d09e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:02.642313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a2110-8258-437a-bccc-afc1b244d09e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:02.642313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a2110-8258-437a-bccc-afc1b244d09e service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:02.651509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1365/2703] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:14:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:14:06.780075 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=20 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:12.706375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=10 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:21.313710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e02f2-4447-4702-9265-42a1ed5191cd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:21.314182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e02f2-4447-4702-9265-42a1ed5191cd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:21.314299 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e02f2-4447-4702-9265-42a1ed5191cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:21.332725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-871e02f2-4447-4702-9265-42a1ed5191cd None None] MFA Rules not processed for user `cc14169c8f7044c6815944af5bc658a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:21.333936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-871e02f2-4447-4702-9265-42a1ed5191cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:21.386745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1339/2704] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:21.649048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a2110-8258-437a-bccc-afc1b244d09e service placement] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:21.712535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a2110-8258-437a-bccc-afc1b244d09e service placement] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:21.717951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:21.718619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:21.718933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:21.719213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] PATH_INFO: `/v3/users/8b333b03b4ab4ef59c492bdd6aa1ae44` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:21.720368 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b333b03b4ab4ef59c492bdd6aa1ae44)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:21.722298 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:21.921091 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b333b03b4ab4ef59c492bdd6aa1ae44'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:21.934001 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b333b03b4ab4ef59c492bdd6aa1ae44'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:21.942055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b333b03b4ab4ef59c492bdd6aa1ae44'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:21.988253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b333b03b4ab4ef59c492bdd6aa1ae44'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:21.988253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b333b03b4ab4ef59c492bdd6aa1ae44'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:21.994155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1366/2705] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:14:21 2022] DELETE /identity/v3/users/8b333b03b4ab4ef59c492bdd6aa1ae44 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:22.003112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea08d-7f78-4080-8e48-8f59d1efbba6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:22.003415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea08d-7f78-4080-8e48-8f59d1efbba6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:22.003765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea08d-7f78-4080-8e48-8f59d1efbba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:22.035139 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:22.036633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:22.048004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ba7ea08d-7f78-4080-8e48-8f59d1efbba6 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:22.049408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ba7ea08d-7f78-4080-8e48-8f59d1efbba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:22.104303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3174cf-3fb4-44a6-a4cf-8e035e651934 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:22.108617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa064dc-fab5-42eb-958a-433c561038f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:22.109231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa064dc-fab5-42eb-958a-433c561038f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:22.109509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa064dc-fab5-42eb-958a-433c561038f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:22.109784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa064dc-fab5-42eb-958a-433c561038f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:22.111672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-caa064dc-fab5-42eb-958a-433c561038f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:22.147501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1340/2706] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:22.152792 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:22.158652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ea08d-7f78-4080-8e48-8f59d1efbba6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:22.168438 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ea08d-7f78-4080-8e48-8f59d1efbba6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:22.173638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425c435-8abf-4abe-b374-b3d2a1ac86aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:22.174251 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425c435-8abf-4abe-b374-b3d2a1ac86aa service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:22.174534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425c435-8abf-4abe-b374-b3d2a1ac86aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:22.179678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425c435-8abf-4abe-b374-b3d2a1ac86aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:22.184658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425c435-8abf-4abe-b374-b3d2a1ac86aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:22.186171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa064dc-fab5-42eb-958a-433c561038f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:22.186657 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425c435-8abf-4abe-b374-b3d2a1ac86aa service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:22.187705 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa064dc-fab5-42eb-958a-433c561038f5 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:22.196460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1341/2707] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:14:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:14:22.200546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1367/2708] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:14:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:14:22.793624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425c435-8abf-4abe-b374-b3d2a1ac86aa service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:22.795198 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8425c435-8abf-4abe-b374-b3d2a1ac86aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:22.820254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425c435-8abf-4abe-b374-b3d2a1ac86aa service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:22.825485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:22.826626 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:22.826961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:22.827236 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] PATH_INFO: `/v3/projects/330421f977444ac9ab30f57e049464a5` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:22.828475 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] RBAC: Authorizing `identity:delete_project(project_id=330421f977444ac9ab30f57e049464a5)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:22.830398 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:22.932794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '330421f977444ac9ab30f57e049464a5', 'request_id': 'req-4712438f-62bf-44e6-9c2a-1ea086afb15f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:22.939847 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '330421f977444ac9ab30f57e049464a5', 'request_id': 'req-4712438f-62bf-44e6-9c2a-1ea086afb15f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:22.945223 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '330421f977444ac9ab30f57e049464a5', 'request_id': 'req-4712438f-62bf-44e6-9c2a-1ea086afb15f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:23.002569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] The token cache is being invalidate because project 330421f977444ac9ab30f57e049464a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:14:23.002969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:23.003893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4712438f-62bf-44e6-9c2a-1ea086afb15f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:23.005562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1342/2709] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:14:22 2022] DELETE /identity/v3/projects/330421f977444ac9ab30f57e049464a5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:23.010598 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d40d7c-d66b-4622-aee5-ff86cc2c44df None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:23.011148 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d40d7c-d66b-4622-aee5-ff86cc2c44df None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:23.011318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d40d7c-d66b-4622-aee5-ff86cc2c44df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:23.033820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-97d40d7c-d66b-4622-aee5-ff86cc2c44df None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:23.034986 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-97d40d7c-d66b-4622-aee5-ff86cc2c44df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:23.098383 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1368/2710] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:14:23.106175 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe0dc5-d294-4efe-a283-bec9aa2a6569 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:23.106526 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe0dc5-d294-4efe-a283-bec9aa2a6569 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:23.106852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe0dc5-d294-4efe-a283-bec9aa2a6569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:23.130540 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9afe0dc5-d294-4efe-a283-bec9aa2a6569 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:23.131950 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9afe0dc5-d294-4efe-a283-bec9aa2a6569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:23.141233 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1343/2711] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:23.145847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d40d7c-d66b-4622-aee5-ff86cc2c44df None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:23.156458 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d40d7c-d66b-4622-aee5-ff86cc2c44df None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:23.161303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2f74-e21c-47bc-880b-d4eabaf6101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:23.161923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2f74-e21c-47bc-880b-d4eabaf6101b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:23.162206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2f74-e21c-47bc-880b-d4eabaf6101b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:23.162480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2f74-e21c-47bc-880b-d4eabaf6101b admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:23.163175 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2f74-e21c-47bc-880b-d4eabaf6101b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:14:23.163486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2f74-e21c-47bc-880b-d4eabaf6101b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:23.165095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2f74-e21c-47bc-880b-d4eabaf6101b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:23.178433 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1369/2712] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:14:23.182733 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe0dc5-d294-4efe-a283-bec9aa2a6569 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:23.193700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe0dc5-d294-4efe-a283-bec9aa2a6569 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:23.211863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0135aa-7317-4b34-8e46-82f21fcc92a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:23.211863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0135aa-7317-4b34-8e46-82f21fcc92a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:23.211863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0135aa-7317-4b34-8e46-82f21fcc92a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:23.211863 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0135aa-7317-4b34-8e46-82f21fcc92a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:23.212270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0135aa-7317-4b34-8e46-82f21fcc92a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:23.212270 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0135aa-7317-4b34-8e46-82f21fcc92a5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:23.217852 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:23.306727 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1344/2713] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:14:23 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:23.318546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2f74-e21c-47bc-880b-d4eabaf6101b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:23.323094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2f74-e21c-47bc-880b-d4eabaf6101b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:23.397127 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9a601-0c17-4c7d-8715-0f4d0de924a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:23.397857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9a601-0c17-4c7d-8715-0f4d0de924a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:23.398138 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9a601-0c17-4c7d-8715-0f4d0de924a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:23.406211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9a601-0c17-4c7d-8715-0f4d0de924a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:23.406211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9a601-0c17-4c7d-8715-0f4d0de924a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:23.406211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9a601-0c17-4c7d-8715-0f4d0de924a5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:23.507923 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1370/2714] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:14:23 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:14:23.512244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0135aa-7317-4b34-8e46-82f21fcc92a5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:23.524283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0135aa-7317-4b34-8e46-82f21fcc92a5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:23.529933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2c233-01ac-49b4-aed3-00e4697b4fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:23.530751 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2c233-01ac-49b4-aed3-00e4697b4fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:23.531036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2c233-01ac-49b4-aed3-00e4697b4fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:23.531443 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2c233-01ac-49b4-aed3-00e4697b4fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:23.532216 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2c233-01ac-49b4-aed3-00e4697b4fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:23.535693 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2c233-01ac-49b4-aed3-00e4697b4fc8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:23.549264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1345/2715] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:14:23.550515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9a601-0c17-4c7d-8715-0f4d0de924a5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:23.562392 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9a601-0c17-4c7d-8715-0f4d0de924a5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:23.567346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d925c4-633a-47d0-b646-0f536a1b810d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:23.569570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d925c4-633a-47d0-b646-0f536a1b810d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:23.569863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d925c4-633a-47d0-b646-0f536a1b810d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:23.570144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d925c4-633a-47d0-b646-0f536a1b810d admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:23.570823 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d925c4-633a-47d0-b646-0f536a1b810d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:23.572424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d925c4-633a-47d0-b646-0f536a1b810d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:23.587474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1371/2716] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:23.589221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2c233-01ac-49b4-aed3-00e4697b4fc8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:23.599885 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2c233-01ac-49b4-aed3-00e4697b4fc8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:23.605644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d8ef7-154e-4096-b77a-db541099c00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:23.606389 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d8ef7-154e-4096-b77a-db541099c00f admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:23.606770 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d8ef7-154e-4096-b77a-db541099c00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:23.607644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d8ef7-154e-4096-b77a-db541099c00f admin admin] PATH_INFO: `/v3/projects/715b5f5887c940aca9d81291d48fd426/users/032d96be1a1146069299d83919adf7ce/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:23.627653 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d8ef7-154e-4096-b77a-db541099c00f admin admin] RBAC: Authorizing `identity:create_grant(project_id=715b5f5887c940aca9d81291d48fd426, user_id=032d96be1a1146069299d83919adf7ce, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:23.630310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d8ef7-154e-4096-b77a-db541099c00f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:23.723759 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1346/2717] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:14:23 2022] PUT /identity/v3/projects/715b5f5887c940aca9d81291d48fd426/users/032d96be1a1146069299d83919adf7ce/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:23.729372 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abbb7d-27d8-4b99-bb33-986e8e48bd08 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:23.729671 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abbb7d-27d8-4b99-bb33-986e8e48bd08 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:23.729945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abbb7d-27d8-4b99-bb33-986e8e48bd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:23.764569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-44abbb7d-27d8-4b99-bb33-986e8e48bd08 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:23.765889 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-44abbb7d-27d8-4b99-bb33-986e8e48bd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:23.799778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:23.834672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1372/2718] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:23.845914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d8ef7-154e-4096-b77a-db541099c00f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:23.847562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-332d8ef7-154e-4096-b77a-db541099c00f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:23.921287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d8ef7-154e-4096-b77a-db541099c00f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:23.925637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3c6e9-fe45-4794-ba38-de7d62a1a3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:23.926262 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3c6e9-fe45-4794-ba38-de7d62a1a3dc service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:23.926545 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3c6e9-fe45-4794-ba38-de7d62a1a3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:23.926870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3c6e9-fe45-4794-ba38-de7d62a1a3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:23.937220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3c6e9-fe45-4794-ba38-de7d62a1a3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:23.938712 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3c6e9-fe45-4794-ba38-de7d62a1a3dc service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:23.950604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1347/2719] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:14:27.031356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44abbb7d-27d8-4b99-bb33-986e8e48bd08 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:27.032826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-44abbb7d-27d8-4b99-bb33-986e8e48bd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:27.046081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44abbb7d-27d8-4b99-bb33-986e8e48bd08 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:27.051445 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08235f-111c-4ff9-aeb6-0ea4637ed896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:27.051915 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08235f-111c-4ff9-aeb6-0ea4637ed896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:27.052635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08235f-111c-4ff9-aeb6-0ea4637ed896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:27.052921 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08235f-111c-4ff9-aeb6-0ea4637ed896 admin admin] PATH_INFO: `/v3/users/81e89f8fb70d49d18cd0c81c98dd4add` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:27.054397 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08235f-111c-4ff9-aeb6-0ea4637ed896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81e89f8fb70d49d18cd0c81c98dd4add)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:27.056258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08235f-111c-4ff9-aeb6-0ea4637ed896 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:27.267874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1d08235f-111c-4ff9-aeb6-0ea4637ed896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81e89f8fb70d49d18cd0c81c98dd4add'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:27.306485 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1d08235f-111c-4ff9-aeb6-0ea4637ed896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81e89f8fb70d49d18cd0c81c98dd4add'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:27.333206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1d08235f-111c-4ff9-aeb6-0ea4637ed896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81e89f8fb70d49d18cd0c81c98dd4add'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:27.420396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1d08235f-111c-4ff9-aeb6-0ea4637ed896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81e89f8fb70d49d18cd0c81c98dd4add'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:27.423684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1d08235f-111c-4ff9-aeb6-0ea4637ed896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81e89f8fb70d49d18cd0c81c98dd4add'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:27.432569 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1373/2720] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:14:27 2022] DELETE /identity/v3/users/81e89f8fb70d49d18cd0c81c98dd4add => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:27.787893 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3c6e9-fe45-4794-ba38-de7d62a1a3dc service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:27.806941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3c6e9-fe45-4794-ba38-de7d62a1a3dc service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:27.909828 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:27.915964 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:27.916308 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:27.917020 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] PATH_INFO: `/v3/projects/8191457181714862bbeb70259fa86971` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:27.921422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8191457181714862bbeb70259fa86971)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:27.924272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:28.111479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8191457181714862bbeb70259fa86971', 'request_id': 'req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:28.120845 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8191457181714862bbeb70259fa86971', 'request_id': 'req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:28.127955 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:28.156206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8191457181714862bbeb70259fa86971', 'request_id': 'req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:28.255299 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] The token cache is being invalidate because project 8191457181714862bbeb70259fa86971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:14:28.255922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:28.256793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b78731ce-5f1e-44dc-ab3e-fabc28ba9a77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:28.264066 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c4cb6-78f6-4fc6-9589-609666583570 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:28.265055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c4cb6-78f6-4fc6-9589-609666583570 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:28.265653 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c4cb6-78f6-4fc6-9589-609666583570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:28.268748 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1348/2721] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:14:27 2022] DELETE /identity/v3/projects/8191457181714862bbeb70259fa86971 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:28.288805 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-056c4cb6-78f6-4fc6-9589-609666583570 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:28.290312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-056c4cb6-78f6-4fc6-9589-609666583570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:28.379429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1374/2722] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:28.388302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54251f-362b-49b5-8739-d409a77f865e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:28.388667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54251f-362b-49b5-8739-d409a77f865e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:28.389027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54251f-362b-49b5-8739-d409a77f865e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:28.419924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ad54251f-362b-49b5-8739-d409a77f865e None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:28.419924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ad54251f-362b-49b5-8739-d409a77f865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:28.432331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1349/2723] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:28.433650 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c4cb6-78f6-4fc6-9589-609666583570 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:28.450401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c4cb6-78f6-4fc6-9589-609666583570 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:28.453636 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ac1be-74a3-4351-95fa-596688f88ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:28.454451 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ac1be-74a3-4351-95fa-596688f88ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:28.454826 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ac1be-74a3-4351-95fa-596688f88ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:28.455351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ac1be-74a3-4351-95fa-596688f88ed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:28.456095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ac1be-74a3-4351-95fa-596688f88ed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:14:28.456478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ac1be-74a3-4351-95fa-596688f88ed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:28.458246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ac1be-74a3-4351-95fa-596688f88ed9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:28.480453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1375/2724] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:14:28.485460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54251f-362b-49b5-8739-d409a77f865e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:28.495337 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54251f-362b-49b5-8739-d409a77f865e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:28.500947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d6643-7ed2-4b4d-ad22-aa3ffe7a48d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:28.501742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d6643-7ed2-4b4d-ad22-aa3ffe7a48d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:28.502092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d6643-7ed2-4b4d-ad22-aa3ffe7a48d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:28.502436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d6643-7ed2-4b4d-ad22-aa3ffe7a48d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:28.503215 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d6643-7ed2-4b4d-ad22-aa3ffe7a48d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:28.505254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d6643-7ed2-4b4d-ad22-aa3ffe7a48d9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:28.627227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ac1be-74a3-4351-95fa-596688f88ed9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:28.628356 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1350/2725] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:14:28 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:28.637748 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ac1be-74a3-4351-95fa-596688f88ed9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:28.677811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:28.709231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7304070-2fcc-4d0a-8e09-ba73c7d8b2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:28.709952 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7304070-2fcc-4d0a-8e09-ba73c7d8b2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:28.710721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7304070-2fcc-4d0a-8e09-ba73c7d8b2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:28.710721 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7304070-2fcc-4d0a-8e09-ba73c7d8b2b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:28.711307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7304070-2fcc-4d0a-8e09-ba73c7d8b2b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:28.713344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7304070-2fcc-4d0a-8e09-ba73c7d8b2b8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:28.834152 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1376/2726] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:14:28 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:28.838518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d6643-7ed2-4b4d-ad22-aa3ffe7a48d9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:28.850250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d6643-7ed2-4b4d-ad22-aa3ffe7a48d9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:28.862020 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acd534-98d3-4991-a7a3-a40a31a4c3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:28.862945 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acd534-98d3-4991-a7a3-a40a31a4c3a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:28.863225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acd534-98d3-4991-a7a3-a40a31a4c3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:28.863500 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acd534-98d3-4991-a7a3-a40a31a4c3a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:28.864707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acd534-98d3-4991-a7a3-a40a31a4c3a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:28.866028 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acd534-98d3-4991-a7a3-a40a31a4c3a2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:28.879926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1351/2727] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:28.881164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7304070-2fcc-4d0a-8e09-ba73c7d8b2b8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:28.893296 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7304070-2fcc-4d0a-8e09-ba73c7d8b2b8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:28.898015 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea67d1d-5181-4949-a877-61d8042b1623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:28.898626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea67d1d-5181-4949-a877-61d8042b1623 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:28.898940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea67d1d-5181-4949-a877-61d8042b1623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:28.899213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea67d1d-5181-4949-a877-61d8042b1623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:28.899857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea67d1d-5181-4949-a877-61d8042b1623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:28.901440 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea67d1d-5181-4949-a877-61d8042b1623 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:28.918120 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1377/2728] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:28.919651 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acd534-98d3-4991-a7a3-a40a31a4c3a2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:28.930654 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acd534-98d3-4991-a7a3-a40a31a4c3a2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:28.935597 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0387b8-d130-4e79-9e6f-05465fe02925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:28.936278 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0387b8-d130-4e79-9e6f-05465fe02925 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:28.936556 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0387b8-d130-4e79-9e6f-05465fe02925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:28.936827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0387b8-d130-4e79-9e6f-05465fe02925 admin admin] PATH_INFO: `/v3/projects/31385ece156c4648a757d94a8e27d594/users/b33ff0a4f1a04b6eb2d4ef9dd7480c49/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:28.953883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0387b8-d130-4e79-9e6f-05465fe02925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31385ece156c4648a757d94a8e27d594, user_id=b33ff0a4f1a04b6eb2d4ef9dd7480c49, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:28.956710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0387b8-d130-4e79-9e6f-05465fe02925 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:29.033156 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1352/2729] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:14:28 2022] PUT /identity/v3/projects/31385ece156c4648a757d94a8e27d594/users/b33ff0a4f1a04b6eb2d4ef9dd7480c49/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:29.043604 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a665d-203b-4240-9120-6d2a118397bc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:29.043817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a665d-203b-4240-9120-6d2a118397bc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:29.043998 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a665d-203b-4240-9120-6d2a118397bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:29.080398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-783a665d-203b-4240-9120-6d2a118397bc None None] MFA Rules not processed for user `b33ff0a4f1a04b6eb2d4ef9dd7480c49`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:29.083617 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-783a665d-203b-4240-9120-6d2a118397bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:29.156481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1378/2730] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:29.169675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0387b8-d130-4e79-9e6f-05465fe02925 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:29.171999 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1b0387b8-d130-4e79-9e6f-05465fe02925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:29.315317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0387b8-d130-4e79-9e6f-05465fe02925 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:29.321816 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7616a3-b4ea-486e-8e96-60857f5fc5fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:29.322465 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7616a3-b4ea-486e-8e96-60857f5fc5fe service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:29.322792 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7616a3-b4ea-486e-8e96-60857f5fc5fe service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:29.323068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7616a3-b4ea-486e-8e96-60857f5fc5fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:29.343996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7616a3-b4ea-486e-8e96-60857f5fc5fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:29.343996 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7616a3-b4ea-486e-8e96-60857f5fc5fe service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:29.351173 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1353/2731] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:14:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:14:30.287266 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e95272-4016-4d70-aca8-b325bf67cc8d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:30.287765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e95272-4016-4d70-aca8-b325bf67cc8d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:30.289231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e95272-4016-4d70-aca8-b325bf67cc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:30.317533 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a2e95272-4016-4d70-aca8-b325bf67cc8d None None] MFA Rules not processed for user `032d96be1a1146069299d83919adf7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:30.318777 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a2e95272-4016-4d70-aca8-b325bf67cc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:30.390943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1379/2732] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:30.396376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7616a3-b4ea-486e-8e96-60857f5fc5fe service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:30.397807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fb7616a3-b4ea-486e-8e96-60857f5fc5fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:30.449739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:30.466738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7616a3-b4ea-486e-8e96-60857f5fc5fe service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:30.471362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bf5bc-962c-4c4e-b3f5-bd7fc47b620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:30.471973 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf5bc-962c-4c4e-b3f5-bd7fc47b620a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:30.472258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf5bc-962c-4c4e-b3f5-bd7fc47b620a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:30.472535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf5bc-962c-4c4e-b3f5-bd7fc47b620a admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:30.473171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bf5bc-962c-4c4e-b3f5-bd7fc47b620a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:30.475143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bf5bc-962c-4c4e-b3f5-bd7fc47b620a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:30.568012 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1354/2733] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:14:30 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:30.571951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e95272-4016-4d70-aca8-b325bf67cc8d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:30.584654 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e95272-4016-4d70-aca8-b325bf67cc8d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:30.617887 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:30.650476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13087d-5913-4d33-9473-741f8fd0cf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:30.651317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13087d-5913-4d33-9473-741f8fd0cf86 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:30.652133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13087d-5913-4d33-9473-741f8fd0cf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:30.652510 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13087d-5913-4d33-9473-741f8fd0cf86 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:30.653371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13087d-5913-4d33-9473-741f8fd0cf86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:30.656033 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13087d-5913-4d33-9473-741f8fd0cf86 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:30.750253 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1380/2734] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:14:30 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:30.751594 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bf5bc-962c-4c4e-b3f5-bd7fc47b620a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:30.763080 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bf5bc-962c-4c4e-b3f5-bd7fc47b620a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:30.768903 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291b42c-f163-4d13-bcfb-f9616e0ca5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:30.769480 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291b42c-f163-4d13-bcfb-f9616e0ca5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:30.769820 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291b42c-f163-4d13-bcfb-f9616e0ca5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:30.770100 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291b42c-f163-4d13-bcfb-f9616e0ca5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:30.770924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291b42c-f163-4d13-bcfb-f9616e0ca5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:30.772631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291b42c-f163-4d13-bcfb-f9616e0ca5a1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:30.785672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1355/2735] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:30.790918 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13087d-5913-4d33-9473-741f8fd0cf86 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:30.802329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13087d-5913-4d33-9473-741f8fd0cf86 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:30.807342 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567755fd-2c4c-44d6-8b84-23cd8dbedcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:30.808025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567755fd-2c4c-44d6-8b84-23cd8dbedcb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:30.808308 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567755fd-2c4c-44d6-8b84-23cd8dbedcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:30.808589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567755fd-2c4c-44d6-8b84-23cd8dbedcb1 admin admin] PATH_INFO: `/v3/projects/05bcf5ce4a1a47778e18c1ddbc7ecf59/users/9b28a738458a4be7bbbf32f111d6e201/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:30.830136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567755fd-2c4c-44d6-8b84-23cd8dbedcb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05bcf5ce4a1a47778e18c1ddbc7ecf59, user_id=9b28a738458a4be7bbbf32f111d6e201, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:30.834047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567755fd-2c4c-44d6-8b84-23cd8dbedcb1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:30.915429 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1381/2736] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:14:30 2022] PUT /identity/v3/projects/05bcf5ce4a1a47778e18c1ddbc7ecf59/users/9b28a738458a4be7bbbf32f111d6e201/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:14:30.915576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291b42c-f163-4d13-bcfb-f9616e0ca5a1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:30.923885 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291b42c-f163-4d13-bcfb-f9616e0ca5a1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:30.987960 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08519021-4a95-42ae-978b-ddf57b6b7a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:30.988582 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08519021-4a95-42ae-978b-ddf57b6b7a7f admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:30.988869 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08519021-4a95-42ae-978b-ddf57b6b7a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:30.989145 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08519021-4a95-42ae-978b-ddf57b6b7a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:30.989831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08519021-4a95-42ae-978b-ddf57b6b7a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:30.991535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08519021-4a95-42ae-978b-ddf57b6b7a7f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:31.002576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1356/2737] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:31.006832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567755fd-2c4c-44d6-8b84-23cd8dbedcb1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:31.018179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567755fd-2c4c-44d6-8b84-23cd8dbedcb1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:31.022919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd99483-d798-4c41-924b-dfcd7cb90c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:31.023677 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd99483-d798-4c41-924b-dfcd7cb90c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:31.024024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd99483-d798-4c41-924b-dfcd7cb90c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:31.024323 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd99483-d798-4c41-924b-dfcd7cb90c5d admin admin] PATH_INFO: `/v3/projects/05bcf5ce4a1a47778e18c1ddbc7ecf59/users/9b28a738458a4be7bbbf32f111d6e201/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:31.026386 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd99483-d798-4c41-924b-dfcd7cb90c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05bcf5ce4a1a47778e18c1ddbc7ecf59, user_id=9b28a738458a4be7bbbf32f111d6e201, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:31.030166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd99483-d798-4c41-924b-dfcd7cb90c5d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:31.041656 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1382/2738] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:14:31 2022] PUT /identity/v3/projects/05bcf5ce4a1a47778e18c1ddbc7ecf59/users/9b28a738458a4be7bbbf32f111d6e201/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:37.166495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b671e76-bc6c-41a1-840b-292202490054 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:37.167136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b671e76-bc6c-41a1-840b-292202490054 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:37.167260 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b671e76-bc6c-41a1-840b-292202490054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:37.197927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4b671e76-bc6c-41a1-840b-292202490054 None None] MFA Rules not processed for user `9b28a738458a4be7bbbf32f111d6e201`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:37.199338 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4b671e76-bc6c-41a1-840b-292202490054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:37.292096 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1357/2739] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:37.306537 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd99483-d798-4c41-924b-dfcd7cb90c5d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:37.421542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd99483-d798-4c41-924b-dfcd7cb90c5d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:37.427690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba8362-6f18-429b-8904-3562535b58da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:37.428308 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba8362-6f18-429b-8904-3562535b58da service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:37.428589 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba8362-6f18-429b-8904-3562535b58da service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:37.428903 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba8362-6f18-429b-8904-3562535b58da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:37.518267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba8362-6f18-429b-8904-3562535b58da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:37.519962 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba8362-6f18-429b-8904-3562535b58da service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:37.533040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1383/2740] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:14:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:14:38.979568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b671e76-bc6c-41a1-840b-292202490054 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:39.027151 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:39.044576 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b671e76-bc6c-41a1-840b-292202490054 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:39.049495 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:39.052305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:39.052305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:39.052305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] PATH_INFO: `/v3/users/b33ff0a4f1a04b6eb2d4ef9dd7480c49` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:39.052305 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b33ff0a4f1a04b6eb2d4ef9dd7480c49)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:39.054047 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:39.236774 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b33ff0a4f1a04b6eb2d4ef9dd7480c49'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:39.251684 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b33ff0a4f1a04b6eb2d4ef9dd7480c49'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:39.256941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b33ff0a4f1a04b6eb2d4ef9dd7480c49'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:39.351519 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b33ff0a4f1a04b6eb2d4ef9dd7480c49'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:39.361517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b33ff0a4f1a04b6eb2d4ef9dd7480c49'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:39.361517 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1358/2741] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:14:38 2022] DELETE /identity/v3/users/b33ff0a4f1a04b6eb2d4ef9dd7480c49 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:39.364991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52f24f-6df9-451a-88f4-7c1ed8dfb18b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:39.365494 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52f24f-6df9-451a-88f4-7c1ed8dfb18b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:39.365620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52f24f-6df9-451a-88f4-7c1ed8dfb18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:39.388035 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cb52f24f-6df9-451a-88f4-7c1ed8dfb18b None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:39.390165 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cb52f24f-6df9-451a-88f4-7c1ed8dfb18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:39.417770 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:39.465935 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1384/2742] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:39.475812 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:39.477422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:39.542549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e7e30-9d77-4794-92cb-0ffcb85eb721 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:39.547725 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68adc364-b0aa-431f-b9c3-c363e3767ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:39.548544 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adc364-b0aa-431f-b9c3-c363e3767ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:39.549000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adc364-b0aa-431f-b9c3-c363e3767ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:39.549455 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adc364-b0aa-431f-b9c3-c363e3767ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:39.561615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68adc364-b0aa-431f-b9c3-c363e3767ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:39.563134 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68adc364-b0aa-431f-b9c3-c363e3767ba6 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:39.574881 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1359/2743] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:14:39.992128 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35450bd-609f-45f9-aa80-b65fc2d0d641 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:39.992565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35450bd-609f-45f9-aa80-b65fc2d0d641 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:39.992939 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35450bd-609f-45f9-aa80-b65fc2d0d641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:40.014246 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f35450bd-609f-45f9-aa80-b65fc2d0d641 None None] MFA Rules not processed for user `032d96be1a1146069299d83919adf7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:40.015667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f35450bd-609f-45f9-aa80-b65fc2d0d641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:40.085641 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68adc364-b0aa-431f-b9c3-c363e3767ba6 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:40.087831 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-68adc364-b0aa-431f-b9c3-c363e3767ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:40.105492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1385/2744] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:40.110916 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68adc364-b0aa-431f-b9c3-c363e3767ba6 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:40.117073 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:40.122021 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:40.122485 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:40.123150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] PATH_INFO: `/v3/projects/31385ece156c4648a757d94a8e27d594` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:40.125688 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31385ece156c4648a757d94a8e27d594)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:40.130595 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:40.258095 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31385ece156c4648a757d94a8e27d594', 'request_id': 'req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:40.266559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31385ece156c4648a757d94a8e27d594', 'request_id': 'req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:40.272745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:40.275185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31385ece156c4648a757d94a8e27d594', 'request_id': 'req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:40.358848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] The token cache is being invalidate because project 31385ece156c4648a757d94a8e27d594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:14:40.359168 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:40.360005 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bbec3c5d-83b0-48c7-b2b6-22a912d9b2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:40.361737 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1360/2745] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:14:40 2022] DELETE /identity/v3/projects/31385ece156c4648a757d94a8e27d594 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:40.366725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470970c1-09b8-4be7-abb7-03eb88caa676 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:40.367258 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470970c1-09b8-4be7-abb7-03eb88caa676 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:40.367371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470970c1-09b8-4be7-abb7-03eb88caa676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:40.391846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-470970c1-09b8-4be7-abb7-03eb88caa676 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:40.393492 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-470970c1-09b8-4be7-abb7-03eb88caa676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:40.461624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1386/2746] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:40.469752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915348d3-f3f5-4d65-b32e-2a223c618c3d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:40.470041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915348d3-f3f5-4d65-b32e-2a223c618c3d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:40.470322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915348d3-f3f5-4d65-b32e-2a223c618c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:40.490864 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-915348d3-f3f5-4d65-b32e-2a223c618c3d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:40.492245 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-915348d3-f3f5-4d65-b32e-2a223c618c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:40.501283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1361/2747] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:40.506024 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470970c1-09b8-4be7-abb7-03eb88caa676 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:40.517022 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470970c1-09b8-4be7-abb7-03eb88caa676 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:40.521817 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590555-e41b-4798-8774-912ae4827e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:40.522432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590555-e41b-4798-8774-912ae4827e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:40.522752 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590555-e41b-4798-8774-912ae4827e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:40.523034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590555-e41b-4798-8774-912ae4827e2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:40.523684 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590555-e41b-4798-8774-912ae4827e2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:14:40.523995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590555-e41b-4798-8774-912ae4827e2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:40.525613 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590555-e41b-4798-8774-912ae4827e2a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:40.538737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1387/2748] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 08:14:40.543187 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915348d3-f3f5-4d65-b32e-2a223c618c3d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:40.554126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915348d3-f3f5-4d65-b32e-2a223c618c3d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:40.563772 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4565d7-564d-4e52-bcf0-8236b4c1c778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:40.564527 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4565d7-564d-4e52-bcf0-8236b4c1c778 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:40.564913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4565d7-564d-4e52-bcf0-8236b4c1c778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:40.565275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4565d7-564d-4e52-bcf0-8236b4c1c778 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:40.566288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4565d7-564d-4e52-bcf0-8236b4c1c778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:40.568392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4565d7-564d-4e52-bcf0-8236b4c1c778 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:40.677499 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1362/2749] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:14:40 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:40.678673 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590555-e41b-4798-8774-912ae4827e2a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:40.690089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590555-e41b-4798-8774-912ae4827e2a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:40.750727 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:40.757894 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fedb6a-c98a-4d8e-9150-ba384a2ba8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:40.758535 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fedb6a-c98a-4d8e-9150-ba384a2ba8ed admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:40.759092 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fedb6a-c98a-4d8e-9150-ba384a2ba8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:40.759205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fedb6a-c98a-4d8e-9150-ba384a2ba8ed admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:40.759765 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fedb6a-c98a-4d8e-9150-ba384a2ba8ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:40.761700 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fedb6a-c98a-4d8e-9150-ba384a2ba8ed admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:40.844754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1388/2750] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:14:40 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:14:40.845917 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4565d7-564d-4e52-bcf0-8236b4c1c778 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:40.868801 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4565d7-564d-4e52-bcf0-8236b4c1c778 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:40.884148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64849899-033e-478b-8206-427621fcaea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:40.885184 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849899-033e-478b-8206-427621fcaea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:40.885460 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849899-033e-478b-8206-427621fcaea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:40.885735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849899-033e-478b-8206-427621fcaea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:40.886379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64849899-033e-478b-8206-427621fcaea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:40.888321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64849899-033e-478b-8206-427621fcaea1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:40.899572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1363/2751] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:40.903820 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fedb6a-c98a-4d8e-9150-ba384a2ba8ed admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:40.929137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fedb6a-c98a-4d8e-9150-ba384a2ba8ed admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:40.942305 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268522b-ff86-4ddc-83af-ec4714505460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:40.943694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268522b-ff86-4ddc-83af-ec4714505460 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:40.944014 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268522b-ff86-4ddc-83af-ec4714505460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:40.944307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268522b-ff86-4ddc-83af-ec4714505460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:40.945209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268522b-ff86-4ddc-83af-ec4714505460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:40.947070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268522b-ff86-4ddc-83af-ec4714505460 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:40.964726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64849899-033e-478b-8206-427621fcaea1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:40.965759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1389/2752] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:40.975436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64849899-033e-478b-8206-427621fcaea1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:40.980419 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaa1fc-b661-4629-a794-317c0cfcd9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:40.981099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaa1fc-b661-4629-a794-317c0cfcd9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:40.981902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaa1fc-b661-4629-a794-317c0cfcd9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:40.982193 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaa1fc-b661-4629-a794-317c0cfcd9c6 admin admin] PATH_INFO: `/v3/projects/f957a51e3a2440728bed76877dbca106/users/b7d7ad4d2f8346b5af8145a7d8a4badd/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:41.004947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecaa1fc-b661-4629-a794-317c0cfcd9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f957a51e3a2440728bed76877dbca106, user_id=b7d7ad4d2f8346b5af8145a7d8a4badd, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:41.007947 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecaa1fc-b661-4629-a794-317c0cfcd9c6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:41.088552 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1364/2753] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:14:40 2022] PUT /identity/v3/projects/f957a51e3a2440728bed76877dbca106/users/b7d7ad4d2f8346b5af8145a7d8a4badd/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:14:41.100432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbc98c-3948-4f3f-a656-e3ba6730399e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:41.100720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbc98c-3948-4f3f-a656-e3ba6730399e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:41.101211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbc98c-3948-4f3f-a656-e3ba6730399e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:41.133749 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-65fbc98c-3948-4f3f-a656-e3ba6730399e None None] MFA Rules not processed for user `b7d7ad4d2f8346b5af8145a7d8a4badd`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:41.135034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-65fbc98c-3948-4f3f-a656-e3ba6730399e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:41.209540 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1390/2754] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:41.213969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaa1fc-b661-4629-a794-317c0cfcd9c6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:41.281486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaa1fc-b661-4629-a794-317c0cfcd9c6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:41.286346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689ded8-ba38-4e61-9909-0992696d5b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:41.287078 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689ded8-ba38-4e61-9909-0992696d5b50 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:41.287465 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689ded8-ba38-4e61-9909-0992696d5b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:41.287628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689ded8-ba38-4e61-9909-0992696d5b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:41.288322 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689ded8-ba38-4e61-9909-0992696d5b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:41.290342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689ded8-ba38-4e61-9909-0992696d5b50 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:41.377749 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1365/2755] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:14:41 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:41.381973 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbc98c-3948-4f3f-a656-e3ba6730399e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:41.399459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbc98c-3948-4f3f-a656-e3ba6730399e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:41.455828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081ce1f-5a7e-4632-a2f5-275aa5c93b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:41.456453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081ce1f-5a7e-4632-a2f5-275aa5c93b29 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:41.456845 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081ce1f-5a7e-4632-a2f5-275aa5c93b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:41.457118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081ce1f-5a7e-4632-a2f5-275aa5c93b29 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:41.457761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081ce1f-5a7e-4632-a2f5-275aa5c93b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:41.459796 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081ce1f-5a7e-4632-a2f5-275aa5c93b29 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:41.548517 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1391/2756] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:14:41 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:41.552605 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689ded8-ba38-4e61-9909-0992696d5b50 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:41.564239 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689ded8-ba38-4e61-9909-0992696d5b50 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:41.569039 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc21668-17dc-4088-b206-40873da308b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:41.569655 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc21668-17dc-4088-b206-40873da308b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:41.574351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc21668-17dc-4088-b206-40873da308b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:41.574351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc21668-17dc-4088-b206-40873da308b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:41.574351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc21668-17dc-4088-b206-40873da308b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:41.574351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc21668-17dc-4088-b206-40873da308b5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:41.585541 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1366/2757] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:41.589893 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081ce1f-5a7e-4632-a2f5-275aa5c93b29 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:41.603353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081ce1f-5a7e-4632-a2f5-275aa5c93b29 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:41.609391 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d22e4-c5a9-463c-8406-ee8f1b465a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:41.610116 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d22e4-c5a9-463c-8406-ee8f1b465a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:41.610474 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d22e4-c5a9-463c-8406-ee8f1b465a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:41.610849 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d22e4-c5a9-463c-8406-ee8f1b465a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:41.611648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d22e4-c5a9-463c-8406-ee8f1b465a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:41.613269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d22e4-c5a9-463c-8406-ee8f1b465a90 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:41.624593 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1392/2758] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:41.628842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc21668-17dc-4088-b206-40873da308b5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:41.640580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc21668-17dc-4088-b206-40873da308b5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:41.645387 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d339684-60f4-47ba-a66a-74e42d44e1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:41.646048 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d339684-60f4-47ba-a66a-74e42d44e1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:41.646349 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d339684-60f4-47ba-a66a-74e42d44e1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:41.646639 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d339684-60f4-47ba-a66a-74e42d44e1f5 admin admin] PATH_INFO: `/v3/projects/97cd00e6d7ce46b1897ac9801a2b7dbf/users/d79588b8b6184234a5f644910d91ec91/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:41.661559 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d339684-60f4-47ba-a66a-74e42d44e1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97cd00e6d7ce46b1897ac9801a2b7dbf, user_id=d79588b8b6184234a5f644910d91ec91, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:41.664293 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d339684-60f4-47ba-a66a-74e42d44e1f5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:41.729501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1367/2759] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:14:41 2022] PUT /identity/v3/projects/97cd00e6d7ce46b1897ac9801a2b7dbf/users/d79588b8b6184234a5f644910d91ec91/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:41.740422 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8bddf-136c-40c0-a4fe-e78f55eef786 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:41.740811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8bddf-136c-40c0-a4fe-e78f55eef786 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:41.741154 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8bddf-136c-40c0-a4fe-e78f55eef786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:41.770281 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-fbb8bddf-136c-40c0-a4fe-e78f55eef786 None None] MFA Rules not processed for user `d79588b8b6184234a5f644910d91ec91`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:41.771544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fbb8bddf-136c-40c0-a4fe-e78f55eef786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:41.804400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:41.839785 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1393/2760] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:41.847154 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d339684-60f4-47ba-a66a-74e42d44e1f5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:41.889976 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:41.939913 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d339684-60f4-47ba-a66a-74e42d44e1f5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:41.945207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f220731-d79a-4641-9b6b-620a4b39cb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:41.945814 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f220731-d79a-4641-9b6b-620a4b39cb40 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:41.946092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f220731-d79a-4641-9b6b-620a4b39cb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:41.946364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f220731-d79a-4641-9b6b-620a4b39cb40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:41.947068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f220731-d79a-4641-9b6b-620a4b39cb40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:41.948966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f220731-d79a-4641-9b6b-620a4b39cb40 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:42.049045 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1368/2761] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:14:41 2022] POST /identity/v3/projects => generated 387 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:42.053204 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8bddf-136c-40c0-a4fe-e78f55eef786 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:42.065144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8bddf-136c-40c0-a4fe-e78f55eef786 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:42.122883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af17e1-4feb-46bb-ad43-c708cec76a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:42.123493 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af17e1-4feb-46bb-ad43-c708cec76a6c admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:42.123769 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af17e1-4feb-46bb-ad43-c708cec76a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:42.124039 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af17e1-4feb-46bb-ad43-c708cec76a6c admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:42.124667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af17e1-4feb-46bb-ad43-c708cec76a6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:42.126616 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af17e1-4feb-46bb-ad43-c708cec76a6c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:42.221167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1394/2762] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:14:42 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:42.225401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f220731-d79a-4641-9b6b-620a4b39cb40 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:42.235761 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f220731-d79a-4641-9b6b-620a4b39cb40 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:42.240367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f5360-65d3-4c72-ae3c-eb43b0cbe812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:42.240966 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f5360-65d3-4c72-ae3c-eb43b0cbe812 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:42.241243 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f5360-65d3-4c72-ae3c-eb43b0cbe812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:42.241516 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f5360-65d3-4c72-ae3c-eb43b0cbe812 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:42.242150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f5360-65d3-4c72-ae3c-eb43b0cbe812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:42.243787 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f5360-65d3-4c72-ae3c-eb43b0cbe812 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:42.253865 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1369/2763] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:42.257646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af17e1-4feb-46bb-ad43-c708cec76a6c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:42.268414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af17e1-4feb-46bb-ad43-c708cec76a6c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:42.273091 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8a7c0-011c-4f0a-a37b-2b662f747af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:42.273761 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8a7c0-011c-4f0a-a37b-2b662f747af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:42.274043 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8a7c0-011c-4f0a-a37b-2b662f747af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:42.274318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8a7c0-011c-4f0a-a37b-2b662f747af1 admin admin] PATH_INFO: `/v3/projects/6a2040fe79a04dc19613b2b32b39d5f8/users/2f9f66a5d02b4b3dbabed46bade70d4e/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:42.291089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8a7c0-011c-4f0a-a37b-2b662f747af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a2040fe79a04dc19613b2b32b39d5f8, user_id=2f9f66a5d02b4b3dbabed46bade70d4e, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:42.293773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8a7c0-011c-4f0a-a37b-2b662f747af1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:42.392289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1395/2764] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:14:42 2022] PUT /identity/v3/projects/6a2040fe79a04dc19613b2b32b39d5f8/users/2f9f66a5d02b4b3dbabed46bade70d4e/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:14:42.396232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f5360-65d3-4c72-ae3c-eb43b0cbe812 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:42.407256 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f5360-65d3-4c72-ae3c-eb43b0cbe812 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:42.463244 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c71222-d7ab-4f26-ba11-095f4eccd7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:42.463849 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71222-d7ab-4f26-ba11-095f4eccd7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:42.464128 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71222-d7ab-4f26-ba11-095f4eccd7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:42.464397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71222-d7ab-4f26-ba11-095f4eccd7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:42.465034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c71222-d7ab-4f26-ba11-095f4eccd7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:42.467325 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c71222-d7ab-4f26-ba11-095f4eccd7a9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:42.477285 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1370/2765] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:42.481452 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8a7c0-011c-4f0a-a37b-2b662f747af1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:42.491596 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8a7c0-011c-4f0a-a37b-2b662f747af1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:42.496149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e4e26-a373-497f-a809-48261c26c6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:42.496794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e4e26-a373-497f-a809-48261c26c6d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:42.497074 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e4e26-a373-497f-a809-48261c26c6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:42.497346 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e4e26-a373-497f-a809-48261c26c6d3 admin admin] PATH_INFO: `/v3/projects/6a2040fe79a04dc19613b2b32b39d5f8/users/2f9f66a5d02b4b3dbabed46bade70d4e/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:42.499317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e4e26-a373-497f-a809-48261c26c6d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a2040fe79a04dc19613b2b32b39d5f8, user_id=2f9f66a5d02b4b3dbabed46bade70d4e, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:42.501905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e4e26-a373-497f-a809-48261c26c6d3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:42.512030 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1396/2766] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:14:42 2022] PUT /identity/v3/projects/6a2040fe79a04dc19613b2b32b39d5f8/users/2f9f66a5d02b4b3dbabed46bade70d4e/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:42.523818 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6bdf3-8b33-4986-988b-68d92a0935a9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:42.524109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6bdf3-8b33-4986-988b-68d92a0935a9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:42.524390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6bdf3-8b33-4986-988b-68d92a0935a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:42.555452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-65f6bdf3-8b33-4986-988b-68d92a0935a9 None None] MFA Rules not processed for user `2f9f66a5d02b4b3dbabed46bade70d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:42.556718 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-65f6bdf3-8b33-4986-988b-68d92a0935a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:42.634824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1371/2767] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:42.646465 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e4e26-a373-497f-a809-48261c26c6d3 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:42.648078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1d8e4e26-a373-497f-a809-48261c26c6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:42.709274 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e4e26-a373-497f-a809-48261c26c6d3 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:42.712961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798385cc-4a53-46d8-b56a-011e813a6e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:42.713564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798385cc-4a53-46d8-b56a-011e813a6e7f service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:42.713870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798385cc-4a53-46d8-b56a-011e813a6e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:42.714144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798385cc-4a53-46d8-b56a-011e813a6e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:42.723625 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798385cc-4a53-46d8-b56a-011e813a6e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:42.725054 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798385cc-4a53-46d8-b56a-011e813a6e7f service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:42.734595 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1397/2768] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:14:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:14:43.780434 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6bdf3-8b33-4986-988b-68d92a0935a9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:43.790211 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6bdf3-8b33-4986-988b-68d92a0935a9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:43.793922 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b5b20-3463-4dee-af35-d6c84cd28cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:43.794490 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b5b20-3463-4dee-af35-d6c84cd28cbf service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:43.794828 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b5b20-3463-4dee-af35-d6c84cd28cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:43.795092 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b5b20-3463-4dee-af35-d6c84cd28cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:43.821094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:43.852610 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b5b20-3463-4dee-af35-d6c84cd28cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:43.854579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b5b20-3463-4dee-af35-d6c84cd28cbf service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:43.864287 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1372/2769] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:14:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:14:48.680615 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798385cc-4a53-46d8-b56a-011e813a6e7f service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:48.682114 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-798385cc-4a53-46d8-b56a-011e813a6e7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:48.693941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798385cc-4a53-46d8-b56a-011e813a6e7f service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:48.697559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd52fe-68de-41ff-b427-d65eab420674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:48.698214 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd52fe-68de-41ff-b427-d65eab420674 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:48.698513 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd52fe-68de-41ff-b427-d65eab420674 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:48.698834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd52fe-68de-41ff-b427-d65eab420674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:48.724328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:48.757645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd52fe-68de-41ff-b427-d65eab420674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:48.759141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd52fe-68de-41ff-b427-d65eab420674 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:48.769612 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1398/2770] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:14:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:14:53.922370 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=15 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:55.683265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b5b20-3463-4dee-af35-d6c84cd28cbf service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:55.744409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b5b20-3463-4dee-af35-d6c84cd28cbf service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:55.749735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:55.750373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:55.750737 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:55.751016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] PATH_INFO: `/v3/users/b7d7ad4d2f8346b5af8145a7d8a4badd` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:55.752140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d7ad4d2f8346b5af8145a7d8a4badd)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:55.754055 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:55.906668 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d7ad4d2f8346b5af8145a7d8a4badd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:55.918314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d7ad4d2f8346b5af8145a7d8a4badd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:55.925633 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d7ad4d2f8346b5af8145a7d8a4badd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:55.996186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d7ad4d2f8346b5af8145a7d8a4badd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:55.996802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d7ad4d2f8346b5af8145a7d8a4badd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:56.003428 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1373/2771] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:14:55 2022] DELETE /identity/v3/users/b7d7ad4d2f8346b5af8145a7d8a4badd => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:56.008110 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3d49f-983e-41e1-83a4-59f22a1fc0df None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:56.008605 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3d49f-983e-41e1-83a4-59f22a1fc0df None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:56.008730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3d49f-983e-41e1-83a4-59f22a1fc0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:56.027876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a0d3d49f-983e-41e1-83a4-59f22a1fc0df None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:56.029136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a0d3d49f-983e-41e1-83a4-59f22a1fc0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:56.086500 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1399/2772] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:56.096827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:56.098207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:56.154931 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458f080-ee7b-479d-bea8-0e0749c727db admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:56.158566 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe231619-1b7d-4633-9b25-a1ae03f900c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:56.159136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231619-1b7d-4633-9b25-a1ae03f900c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:56.159420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231619-1b7d-4633-9b25-a1ae03f900c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:56.159689 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231619-1b7d-4633-9b25-a1ae03f900c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:56.168508 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe231619-1b7d-4633-9b25-a1ae03f900c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:56.169941 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe231619-1b7d-4633-9b25-a1ae03f900c1 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:56.179904 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1374/2773] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:14:56.569424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3d49f-983e-41e1-83a4-59f22a1fc0df None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:56.570892 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a0d3d49f-983e-41e1-83a4-59f22a1fc0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:56.583528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3d49f-983e-41e1-83a4-59f22a1fc0df None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:56.588089 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:56.588738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:56.589003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:56.589306 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] PATH_INFO: `/v3/projects/f957a51e3a2440728bed76877dbca106` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:56.590443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f957a51e3a2440728bed76877dbca106)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:56.593044 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:56.699609 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f957a51e3a2440728bed76877dbca106', 'request_id': 'req-afc1f03d-72a8-4353-8162-f5058d288da9'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:56.706572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f957a51e3a2440728bed76877dbca106', 'request_id': 'req-afc1f03d-72a8-4353-8162-f5058d288da9'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:56.712331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f957a51e3a2440728bed76877dbca106', 'request_id': 'req-afc1f03d-72a8-4353-8162-f5058d288da9'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:56.773610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] The token cache is being invalidate because project f957a51e3a2440728bed76877dbca106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:14:56.773902 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:56.774516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:56.776037 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1400/2774] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:14:56 2022] DELETE /identity/v3/projects/f957a51e3a2440728bed76877dbca106 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:56.780981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe231619-1b7d-4633-9b25-a1ae03f900c1 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:56.790604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe231619-1b7d-4633-9b25-a1ae03f900c1 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:56.808099 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:56.847867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:56.848497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:56.848794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:56.849058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] PATH_INFO: `/v3/users/d79588b8b6184234a5f644910d91ec91` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:56.850183 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d79588b8b6184234a5f644910d91ec91)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:56.852234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:57.061392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd79588b8b6184234a5f644910d91ec91'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:57.073103 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd79588b8b6184234a5f644910d91ec91'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:57.080796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd79588b8b6184234a5f644910d91ec91'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:57.145031 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd79588b8b6184234a5f644910d91ec91'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:57.145638 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd79588b8b6184234a5f644910d91ec91'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:57.152156 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1375/2775] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:14:56 2022] DELETE /identity/v3/users/d79588b8b6184234a5f644910d91ec91 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:14:57.563725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:57.565977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:57.580221 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1f03d-72a8-4353-8162-f5058d288da9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:57.626762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:57.638957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:57.644279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:57.644279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:57.644279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] PATH_INFO: `/v3/projects/97cd00e6d7ce46b1897ac9801a2b7dbf` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:57.644279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] RBAC: Authorizing `identity:delete_project(project_id=97cd00e6d7ce46b1897ac9801a2b7dbf)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:57.644279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:57.734091 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97cd00e6d7ce46b1897ac9801a2b7dbf', 'request_id': 'req-721732bd-98bf-4968-8675-40564358e42e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:57.741322 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97cd00e6d7ce46b1897ac9801a2b7dbf', 'request_id': 'req-721732bd-98bf-4968-8675-40564358e42e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:57.746858 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97cd00e6d7ce46b1897ac9801a2b7dbf', 'request_id': 'req-721732bd-98bf-4968-8675-40564358e42e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:57.843375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] The token cache is being invalidate because project 97cd00e6d7ce46b1897ac9801a2b7dbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:14:57.843688 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:57.844425 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:57.846063 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1401/2776] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:14:57 2022] DELETE /identity/v3/projects/97cd00e6d7ce46b1897ac9801a2b7dbf => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:57.850242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:57.851811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:57.865694 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93922cb5-20ac-439d-b04d-e31873a6da04 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:57.926630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc98040-d8d2-4512-98bd-03854a1761c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:57.927350 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc98040-d8d2-4512-98bd-03854a1761c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:57.927635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc98040-d8d2-4512-98bd-03854a1761c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:57.927915 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc98040-d8d2-4512-98bd-03854a1761c4 admin admin] PATH_INFO: `/v3/users/2f9f66a5d02b4b3dbabed46bade70d4e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:57.929077 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc98040-d8d2-4512-98bd-03854a1761c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9f66a5d02b4b3dbabed46bade70d4e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:57.931106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc98040-d8d2-4512-98bd-03854a1761c4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:58.123089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:58.125562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2fc98040-d8d2-4512-98bd-03854a1761c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9f66a5d02b4b3dbabed46bade70d4e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:58.137486 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2fc98040-d8d2-4512-98bd-03854a1761c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9f66a5d02b4b3dbabed46bade70d4e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:58.144686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2fc98040-d8d2-4512-98bd-03854a1761c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9f66a5d02b4b3dbabed46bade70d4e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:58.196166 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2fc98040-d8d2-4512-98bd-03854a1761c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9f66a5d02b4b3dbabed46bade70d4e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:58.196791 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2fc98040-d8d2-4512-98bd-03854a1761c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9f66a5d02b4b3dbabed46bade70d4e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:58.203414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1376/2777] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:14:57 2022] DELETE /identity/v3/users/2f9f66a5d02b4b3dbabed46bade70d4e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:58.569907 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:58.571424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:58.588457 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721732bd-98bf-4968-8675-40564358e42e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:58.646714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:58.647277 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:58.647559 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:58.647834 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] PATH_INFO: `/v3/projects/6a2040fe79a04dc19613b2b32b39d5f8` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:58.648948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a2040fe79a04dc19613b2b32b39d5f8)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:58.650888 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:58.752328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a2040fe79a04dc19613b2b32b39d5f8', 'request_id': 'req-bafd85d1-7ee8-42df-ae90-4959890966e5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:58.758810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a2040fe79a04dc19613b2b32b39d5f8', 'request_id': 'req-bafd85d1-7ee8-42df-ae90-4959890966e5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:58.764447 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a2040fe79a04dc19613b2b32b39d5f8', 'request_id': 'req-bafd85d1-7ee8-42df-ae90-4959890966e5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:58.869695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] The token cache is being invalidate because project 6a2040fe79a04dc19613b2b32b39d5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:14:58.869968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:58.870592 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bafd85d1-7ee8-42df-ae90-4959890966e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:14:58.872113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1402/2778] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:14:58 2022] DELETE /identity/v3/projects/6a2040fe79a04dc19613b2b32b39d5f8 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:58.878832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f0cf3-76b1-4219-ab33-74811e015d4c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:58.879308 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f0cf3-76b1-4219-ab33-74811e015d4c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:58.879409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f0cf3-76b1-4219-ab33-74811e015d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:58.898908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-058f0cf3-76b1-4219-ab33-74811e015d4c None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:58.900058 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-058f0cf3-76b1-4219-ab33-74811e015d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:58.959706 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1377/2779] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:58.968341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0db0a1-9b2a-45e5-866d-3b4a20032b0a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:58.968627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0db0a1-9b2a-45e5-866d-3b4a20032b0a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:58.968906 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0db0a1-9b2a-45e5-866d-3b4a20032b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:58.992181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1b0db0a1-9b2a-45e5-866d-3b4a20032b0a None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:58.992181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1b0db0a1-9b2a-45e5-866d-3b4a20032b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:58.999209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1403/2780] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:59.004110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f0cf3-76b1-4219-ab33-74811e015d4c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:59.014068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f0cf3-76b1-4219-ab33-74811e015d4c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:59.018973 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a2750-71d7-4cdf-8df2-912c027457b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:59.019736 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a2750-71d7-4cdf-8df2-912c027457b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:59.020024 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a2750-71d7-4cdf-8df2-912c027457b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:59.020303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a2750-71d7-4cdf-8df2-912c027457b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:59.020957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a2750-71d7-4cdf-8df2-912c027457b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:14:59.021265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a2750-71d7-4cdf-8df2-912c027457b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:59.022877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a2750-71d7-4cdf-8df2-912c027457b1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:59.035421 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1378/2781] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:14:59.039807 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0db0a1-9b2a-45e5-866d-3b4a20032b0a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:59.044321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:59.050156 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0db0a1-9b2a-45e5-866d-3b4a20032b0a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:59.054871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d2fc6-fb4f-4768-87c2-f8fb5e665942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:59.058524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d2fc6-fb4f-4768-87c2-f8fb5e665942 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:59.058524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d2fc6-fb4f-4768-87c2-f8fb5e665942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:59.058524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d2fc6-fb4f-4768-87c2-f8fb5e665942 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:59.058524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d2fc6-fb4f-4768-87c2-f8fb5e665942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:59.058524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d2fc6-fb4f-4768-87c2-f8fb5e665942 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:59.134437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1404/2782] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:14:59 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:14:59.138811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a2750-71d7-4cdf-8df2-912c027457b1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:59.148836 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a2750-71d7-4cdf-8df2-912c027457b1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:59.204442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405f7c2-783d-4c2d-ad0b-370c523646d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:59.205040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405f7c2-783d-4c2d-ad0b-370c523646d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:59.205317 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405f7c2-783d-4c2d-ad0b-370c523646d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:59.205587 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405f7c2-783d-4c2d-ad0b-370c523646d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:59.206221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405f7c2-783d-4c2d-ad0b-370c523646d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:59.208246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405f7c2-783d-4c2d-ad0b-370c523646d0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:59.276041 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1379/2783] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:14:59 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:59.280466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d2fc6-fb4f-4768-87c2-f8fb5e665942 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:59.291580 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d2fc6-fb4f-4768-87c2-f8fb5e665942 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:59.296223 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae0fdf-4682-45fe-a547-467fea7fd5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:59.297388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0fdf-4682-45fe-a547-467fea7fd5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:59.300404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0fdf-4682-45fe-a547-467fea7fd5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:59.300404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0fdf-4682-45fe-a547-467fea7fd5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:59.300404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae0fdf-4682-45fe-a547-467fea7fd5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:59.300404 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae0fdf-4682-45fe-a547-467fea7fd5a4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:59.310575 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1405/2784] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:59.314437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405f7c2-783d-4c2d-ad0b-370c523646d0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:59.324451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405f7c2-783d-4c2d-ad0b-370c523646d0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:59.329283 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f26fa9-9338-483e-86d9-7f295ef4a046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:59.329874 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f26fa9-9338-483e-86d9-7f295ef4a046 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:59.330152 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f26fa9-9338-483e-86d9-7f295ef4a046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:59.330424 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f26fa9-9338-483e-86d9-7f295ef4a046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:59.331102 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f26fa9-9338-483e-86d9-7f295ef4a046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:59.332762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f26fa9-9338-483e-86d9-7f295ef4a046 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:59.344042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1380/2785] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:59.348087 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae0fdf-4682-45fe-a547-467fea7fd5a4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:59.357897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae0fdf-4682-45fe-a547-467fea7fd5a4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:59.363153 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec410e0c-6043-45a0-9312-b18c13c7d25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:59.363799 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec410e0c-6043-45a0-9312-b18c13c7d25e admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:59.364080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec410e0c-6043-45a0-9312-b18c13c7d25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:59.364356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec410e0c-6043-45a0-9312-b18c13c7d25e admin admin] PATH_INFO: `/v3/projects/2c5f770ecd0944f482ed2d575cb989e7/users/924fea555cd74d70be77c7f4f029362f/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:59.378515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec410e0c-6043-45a0-9312-b18c13c7d25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5f770ecd0944f482ed2d575cb989e7, user_id=924fea555cd74d70be77c7f4f029362f, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:59.381216 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec410e0c-6043-45a0-9312-b18c13c7d25e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:59.424470 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1406/2786] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:14:59 2022] PUT /identity/v3/projects/2c5f770ecd0944f482ed2d575cb989e7/users/924fea555cd74d70be77c7f4f029362f/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:14:59.435715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd7ece-f05b-4f30-8c17-f08bf64f37d4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:59.436005 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd7ece-f05b-4f30-8c17-f08bf64f37d4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:59.436303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd7ece-f05b-4f30-8c17-f08bf64f37d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:59.464321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-29dd7ece-f05b-4f30-8c17-f08bf64f37d4 None None] MFA Rules not processed for user `924fea555cd74d70be77c7f4f029362f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:14:59.465475 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-29dd7ece-f05b-4f30-8c17-f08bf64f37d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:14:59.478489 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:14:59.529150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1381/2787] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:14:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:14:59.534558 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec410e0c-6043-45a0-9312-b18c13c7d25e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:59.600133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec410e0c-6043-45a0-9312-b18c13c7d25e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:59.605462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e918e0-1137-4473-85e0-d1c5e5dd78c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:59.606068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e918e0-1137-4473-85e0-d1c5e5dd78c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:59.606356 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e918e0-1137-4473-85e0-d1c5e5dd78c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:59.606631 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e918e0-1137-4473-85e0-d1c5e5dd78c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:59.607289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e918e0-1137-4473-85e0-d1c5e5dd78c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:59.609181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e918e0-1137-4473-85e0-d1c5e5dd78c7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:59.690025 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1407/2788] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:14:59 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:59.694484 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd7ece-f05b-4f30-8c17-f08bf64f37d4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:59.705895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd7ece-f05b-4f30-8c17-f08bf64f37d4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:59.761740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182c5bf-2b08-4e6f-bb4f-7995670dda1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:59.762343 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182c5bf-2b08-4e6f-bb4f-7995670dda1b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:59.762620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182c5bf-2b08-4e6f-bb4f-7995670dda1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:59.762921 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182c5bf-2b08-4e6f-bb4f-7995670dda1b admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:59.763551 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182c5bf-2b08-4e6f-bb4f-7995670dda1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:59.766070 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182c5bf-2b08-4e6f-bb4f-7995670dda1b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:59.853624 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1382/2789] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:14:59 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:14:59.857674 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e918e0-1137-4473-85e0-d1c5e5dd78c7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:59.868125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e918e0-1137-4473-85e0-d1c5e5dd78c7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:59.872836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd1c55-1a06-4301-8c9a-7dc8b0af709b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:59.873434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd1c55-1a06-4301-8c9a-7dc8b0af709b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:59.873719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd1c55-1a06-4301-8c9a-7dc8b0af709b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:59.873997 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd1c55-1a06-4301-8c9a-7dc8b0af709b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:59.875224 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd1c55-1a06-4301-8c9a-7dc8b0af709b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:59.876848 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd1c55-1a06-4301-8c9a-7dc8b0af709b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:59.887615 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1408/2790] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:14:59.892351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182c5bf-2b08-4e6f-bb4f-7995670dda1b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:14:59.902889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182c5bf-2b08-4e6f-bb4f-7995670dda1b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:14:59.907604 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe0f06-e595-465e-b074-337f186038ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:14:59.908246 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe0f06-e595-465e-b074-337f186038ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:14:59.908525 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe0f06-e595-465e-b074-337f186038ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:14:59.908798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe0f06-e595-465e-b074-337f186038ce admin admin] PATH_INFO: `/v3/projects/9b0b863a6aed4a60a4ac48b3b8173f2d/users/0588e570fd31439aac8145a0a5c7f7b1/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:14:59.923209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe0f06-e595-465e-b074-337f186038ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b0b863a6aed4a60a4ac48b3b8173f2d, user_id=0588e570fd31439aac8145a0a5c7f7b1, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:14:59.925856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe0f06-e595-465e-b074-337f186038ce admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:14:59.985040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1383/2791] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:14:59 2022] PUT /identity/v3/projects/9b0b863a6aed4a60a4ac48b3b8173f2d/users/0588e570fd31439aac8145a0a5c7f7b1/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:14:59.990343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd1c55-1a06-4301-8c9a-7dc8b0af709b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:00.001171 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd1c55-1a06-4301-8c9a-7dc8b0af709b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:00.059759 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4ebf-a6d5-4915-a303-a5eb48cc1987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:00.060374 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4ebf-a6d5-4915-a303-a5eb48cc1987 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:00.063081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4ebf-a6d5-4915-a303-a5eb48cc1987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:00.063081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4ebf-a6d5-4915-a303-a5eb48cc1987 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:00.063081 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4ebf-a6d5-4915-a303-a5eb48cc1987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:00.063791 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4ebf-a6d5-4915-a303-a5eb48cc1987 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:00.076991 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1409/2792] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:00.078804 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe0f06-e595-465e-b074-337f186038ce admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:00.091773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe0f06-e595-465e-b074-337f186038ce admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:00.096464 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bd664-8cde-4cca-8b51-cb105777c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:00.097110 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bd664-8cde-4cca-8b51-cb105777c1d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:00.097430 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bd664-8cde-4cca-8b51-cb105777c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:00.097719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bd664-8cde-4cca-8b51-cb105777c1d7 admin admin] PATH_INFO: `/v3/projects/9b0b863a6aed4a60a4ac48b3b8173f2d/users/0588e570fd31439aac8145a0a5c7f7b1/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:00.099783 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bd664-8cde-4cca-8b51-cb105777c1d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b0b863a6aed4a60a4ac48b3b8173f2d, user_id=0588e570fd31439aac8145a0a5c7f7b1, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:00.102475 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bd664-8cde-4cca-8b51-cb105777c1d7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:00.114235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1384/2793] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:15:00 2022] PUT /identity/v3/projects/9b0b863a6aed4a60a4ac48b3b8173f2d/users/0588e570fd31439aac8145a0a5c7f7b1/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:00.124181 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accd199-c41e-40b0-bdaf-6868a035abd3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:00.125136 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accd199-c41e-40b0-bdaf-6868a035abd3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:00.125419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accd199-c41e-40b0-bdaf-6868a035abd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:00.153199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8accd199-c41e-40b0-bdaf-6868a035abd3 None None] MFA Rules not processed for user `0588e570fd31439aac8145a0a5c7f7b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:00.154413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8accd199-c41e-40b0-bdaf-6868a035abd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:00.158343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:00.222362 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1410/2794] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:15:00.233033 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bd664-8cde-4cca-8b51-cb105777c1d7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:00.235288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-964bd664-8cde-4cca-8b51-cb105777c1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:00.294194 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bd664-8cde-4cca-8b51-cb105777c1d7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:00.298229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aefeb3-129f-4b13-b0ec-567c43cd8d8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:00.298851 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aefeb3-129f-4b13-b0ec-567c43cd8d8a service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:00.299129 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aefeb3-129f-4b13-b0ec-567c43cd8d8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:00.299400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aefeb3-129f-4b13-b0ec-567c43cd8d8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:00.308226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aefeb3-129f-4b13-b0ec-567c43cd8d8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:00.309648 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aefeb3-129f-4b13-b0ec-567c43cd8d8a service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:00.319574 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1385/2795] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:00.428240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accd199-c41e-40b0-bdaf-6868a035abd3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:00.438778 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accd199-c41e-40b0-bdaf-6868a035abd3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:00.442408 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3f019-42b6-4d60-99b3-3a409ea23c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:00.443093 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3f019-42b6-4d60-99b3-3a409ea23c34 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:00.443375 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3f019-42b6-4d60-99b3-3a409ea23c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:00.443650 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3f019-42b6-4d60-99b3-3a409ea23c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:00.498648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3f019-42b6-4d60-99b3-3a409ea23c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:00.500720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3f019-42b6-4d60-99b3-3a409ea23c34 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:00.509456 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1411/2796] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:03.757414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aefeb3-129f-4b13-b0ec-567c43cd8d8a service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:03.769115 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:03.821034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aefeb3-129f-4b13-b0ec-567c43cd8d8a service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:03.825892 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:03.826535 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:03.826856 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:03.827136 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] PATH_INFO: `/v3/users/924fea555cd74d70be77c7f4f029362f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:03.828265 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=924fea555cd74d70be77c7f4f029362f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:03.830253 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:04.003637 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '924fea555cd74d70be77c7f4f029362f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:04.015802 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '924fea555cd74d70be77c7f4f029362f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:04.022652 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '924fea555cd74d70be77c7f4f029362f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:04.094094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '924fea555cd74d70be77c7f4f029362f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:04.094634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '924fea555cd74d70be77c7f4f029362f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:04.100900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1386/2797] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:15:03 2022] DELETE /identity/v3/users/924fea555cd74d70be77c7f4f029362f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:04.105469 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02259-2f2a-4011-bbdc-facc671fc6f0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:04.105961 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02259-2f2a-4011-bbdc-facc671fc6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:04.106076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe02259-2f2a-4011-bbdc-facc671fc6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:04.124974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1fe02259-2f2a-4011-bbdc-facc671fc6f0 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:04.126216 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1fe02259-2f2a-4011-bbdc-facc671fc6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:04.185197 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1412/2798] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:04.196134 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:04.197509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:04.259469 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec36838-9549-4eb1-a2c5-cf4d860f29d5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:04.263328 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a126791-b7b9-4812-a8d4-bfbca9938077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:04.264416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a126791-b7b9-4812-a8d4-bfbca9938077 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:04.264416 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a126791-b7b9-4812-a8d4-bfbca9938077 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:04.264593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a126791-b7b9-4812-a8d4-bfbca9938077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:04.274620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a126791-b7b9-4812-a8d4-bfbca9938077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:04.276093 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a126791-b7b9-4812-a8d4-bfbca9938077 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:04.287329 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1387/2799] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:15:04.703435 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe02259-2f2a-4011-bbdc-facc671fc6f0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:04.704256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1fe02259-2f2a-4011-bbdc-facc671fc6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:04.732438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe02259-2f2a-4011-bbdc-facc671fc6f0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:04.733757 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:04.735113 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:04.735394 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:04.735667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] PATH_INFO: `/v3/projects/2c5f770ecd0944f482ed2d575cb989e7` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:04.736859 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c5f770ecd0944f482ed2d575cb989e7)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:04.738880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:04.829482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:04.838946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c5f770ecd0944f482ed2d575cb989e7', 'request_id': 'req-31b869fe-39eb-4163-b855-26dc00df2aac'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:04.845514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c5f770ecd0944f482ed2d575cb989e7', 'request_id': 'req-31b869fe-39eb-4163-b855-26dc00df2aac'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:04.851341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c5f770ecd0944f482ed2d575cb989e7', 'request_id': 'req-31b869fe-39eb-4163-b855-26dc00df2aac'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:04.916498 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] The token cache is being invalidate because project 2c5f770ecd0944f482ed2d575cb989e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:15:04.918196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:04.918196 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:04.919095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1413/2800] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:15:04 2022] DELETE /identity/v3/projects/2c5f770ecd0944f482ed2d575cb989e7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:04.923659 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a126791-b7b9-4812-a8d4-bfbca9938077 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:04.935452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a126791-b7b9-4812-a8d4-bfbca9938077 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:04.965635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:04.992060 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431c6b6-abcc-455a-8a38-625520f85f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:04.993140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c6b6-abcc-455a-8a38-625520f85f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:04.993451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c6b6-abcc-455a-8a38-625520f85f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:04.993866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c6b6-abcc-455a-8a38-625520f85f11 admin admin] PATH_INFO: `/v3/users/0588e570fd31439aac8145a0a5c7f7b1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:04.995121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431c6b6-abcc-455a-8a38-625520f85f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0588e570fd31439aac8145a0a5c7f7b1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:05.003197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431c6b6-abcc-455a-8a38-625520f85f11 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:05.165081 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9431c6b6-abcc-455a-8a38-625520f85f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0588e570fd31439aac8145a0a5c7f7b1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:05.177925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9431c6b6-abcc-455a-8a38-625520f85f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0588e570fd31439aac8145a0a5c7f7b1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:05.184866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9431c6b6-abcc-455a-8a38-625520f85f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0588e570fd31439aac8145a0a5c7f7b1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:05.241807 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9431c6b6-abcc-455a-8a38-625520f85f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0588e570fd31439aac8145a0a5c7f7b1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:05.242362 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9431c6b6-abcc-455a-8a38-625520f85f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0588e570fd31439aac8145a0a5c7f7b1'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:05.248726 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1388/2801] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:15:04 2022] DELETE /identity/v3/users/0588e570fd31439aac8145a0a5c7f7b1 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:05.620285 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:05.621833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:05.636855 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b869fe-39eb-4163-b855-26dc00df2aac admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:05.693954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:05.694587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:05.694899 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:05.695177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] PATH_INFO: `/v3/projects/9b0b863a6aed4a60a4ac48b3b8173f2d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:05.696303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b0b863a6aed4a60a4ac48b3b8173f2d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:05.698200 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:05.794151 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b0b863a6aed4a60a4ac48b3b8173f2d', 'request_id': 'req-d6194aa6-9aad-4db6-961b-8a6b90b36a94'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:05.801186 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b0b863a6aed4a60a4ac48b3b8173f2d', 'request_id': 'req-d6194aa6-9aad-4db6-961b-8a6b90b36a94'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:05.807682 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b0b863a6aed4a60a4ac48b3b8173f2d', 'request_id': 'req-d6194aa6-9aad-4db6-961b-8a6b90b36a94'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:05.882053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] The token cache is being invalidate because project 9b0b863a6aed4a60a4ac48b3b8173f2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:15:05.882341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:05.882974 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6194aa6-9aad-4db6-961b-8a6b90b36a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:05.884438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1414/2802] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:15:05 2022] DELETE /identity/v3/projects/9b0b863a6aed4a60a4ac48b3b8173f2d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:15:05.889232 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69b568-67a0-4385-b847-0aefdac571aa None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:05.889630 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69b568-67a0-4385-b847-0aefdac571aa None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:05.889909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69b568-67a0-4385-b847-0aefdac571aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:05.910873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9c69b568-67a0-4385-b847-0aefdac571aa None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:05.912087 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9c69b568-67a0-4385-b847-0aefdac571aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:05.971005 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1389/2803] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:05.978821 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632bdf2-90db-49d6-b568-b62fdc686a48 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:05.979111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632bdf2-90db-49d6-b568-b62fdc686a48 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:05.979388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632bdf2-90db-49d6-b568-b62fdc686a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:05.999951 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7632bdf2-90db-49d6-b568-b62fdc686a48 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:06.001178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7632bdf2-90db-49d6-b568-b62fdc686a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:06.009096 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1415/2804] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:06.013373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69b568-67a0-4385-b847-0aefdac571aa None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:06.023397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69b568-67a0-4385-b847-0aefdac571aa None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:06.028302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de5cc3-62bf-49a2-873a-d0101fba05d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:06.028785 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de5cc3-62bf-49a2-873a-d0101fba05d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:06.029064 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de5cc3-62bf-49a2-873a-d0101fba05d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:06.029347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de5cc3-62bf-49a2-873a-d0101fba05d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:06.029997 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de5cc3-62bf-49a2-873a-d0101fba05d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:15:06.030297 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de5cc3-62bf-49a2-873a-d0101fba05d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:06.032596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de5cc3-62bf-49a2-873a-d0101fba05d9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:06.044662 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1390/2805] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:15:06.049598 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632bdf2-90db-49d6-b568-b62fdc686a48 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:06.058871 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632bdf2-90db-49d6-b568-b62fdc686a48 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:06.063565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0a5fe-9abf-4ee6-9aef-92e8bd0a3d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:06.064472 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0a5fe-9abf-4ee6-9aef-92e8bd0a3d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:06.064472 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0a5fe-9abf-4ee6-9aef-92e8bd0a3d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:06.064729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0a5fe-9abf-4ee6-9aef-92e8bd0a3d72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:06.065328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0a5fe-9abf-4ee6-9aef-92e8bd0a3d72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:06.067269 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0a5fe-9abf-4ee6-9aef-92e8bd0a3d72 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:06.180369 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1416/2806] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:15:06 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:15:06.187742 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de5cc3-62bf-49a2-873a-d0101fba05d9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:06.197974 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de5cc3-62bf-49a2-873a-d0101fba05d9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:06.236813 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:06.255743 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50ee4d-3800-485e-acc2-c9715858ba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:06.256347 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50ee4d-3800-485e-acc2-c9715858ba22 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:06.256627 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50ee4d-3800-485e-acc2-c9715858ba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:06.256902 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50ee4d-3800-485e-acc2-c9715858ba22 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:06.257600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50ee4d-3800-485e-acc2-c9715858ba22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:06.259601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50ee4d-3800-485e-acc2-c9715858ba22 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:06.350908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1391/2807] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:15:06 2022] POST /identity/v3/users => generated 290 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:06.354853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0a5fe-9abf-4ee6-9aef-92e8bd0a3d72 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:06.365668 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0a5fe-9abf-4ee6-9aef-92e8bd0a3d72 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:06.370311 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c24ebd-6fcd-48e2-ba0d-e8903f7c0ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:06.370943 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c24ebd-6fcd-48e2-ba0d-e8903f7c0ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:06.371227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c24ebd-6fcd-48e2-ba0d-e8903f7c0ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:06.371507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c24ebd-6fcd-48e2-ba0d-e8903f7c0ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:06.372310 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c24ebd-6fcd-48e2-ba0d-e8903f7c0ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:06.373901 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c24ebd-6fcd-48e2-ba0d-e8903f7c0ad5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:06.384459 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1417/2808] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:15:06.388351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50ee4d-3800-485e-acc2-c9715858ba22 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:06.398410 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50ee4d-3800-485e-acc2-c9715858ba22 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:06.407318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a409c-e8bd-4353-980f-51f650b0a7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:06.407318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a409c-e8bd-4353-980f-51f650b0a7fd admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:06.407318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a409c-e8bd-4353-980f-51f650b0a7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:06.407318 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a409c-e8bd-4353-980f-51f650b0a7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:06.407716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a409c-e8bd-4353-980f-51f650b0a7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:06.407716 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a409c-e8bd-4353-980f-51f650b0a7fd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:06.417186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1392/2809] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:06.421125 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c24ebd-6fcd-48e2-ba0d-e8903f7c0ad5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:06.431547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c24ebd-6fcd-48e2-ba0d-e8903f7c0ad5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:06.436345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ce420-4385-4d7a-81e4-5d8c1fe07e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:06.436995 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce420-4385-4d7a-81e4-5d8c1fe07e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:06.437278 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce420-4385-4d7a-81e4-5d8c1fe07e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:06.437560 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce420-4385-4d7a-81e4-5d8c1fe07e1f admin admin] PATH_INFO: `/v3/projects/a8caae866029435bb715f06699a6da46/users/5cb98dd23fe945b8b053b9d6e498a900/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:06.452047 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:06.455217 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ce420-4385-4d7a-81e4-5d8c1fe07e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8caae866029435bb715f06699a6da46, user_id=5cb98dd23fe945b8b053b9d6e498a900, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:06.457866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ce420-4385-4d7a-81e4-5d8c1fe07e1f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:06.569539 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1418/2810] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:15:06 2022] PUT /identity/v3/projects/a8caae866029435bb715f06699a6da46/users/5cb98dd23fe945b8b053b9d6e498a900/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:06.635271 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f9a96-5be2-4980-baf7-2f8c0c7de9e7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:06.635507 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f9a96-5be2-4980-baf7-2f8c0c7de9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:06.635762 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f9a96-5be2-4980-baf7-2f8c0c7de9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:06.663892 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5c8f9a96-5be2-4980-baf7-2f8c0c7de9e7 None None] MFA Rules not processed for user `5cb98dd23fe945b8b053b9d6e498a900`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:06.665132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5c8f9a96-5be2-4980-baf7-2f8c0c7de9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:06.731563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1393/2811] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:06.736023 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ce420-4385-4d7a-81e4-5d8c1fe07e1f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:06.798524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ce420-4385-4d7a-81e4-5d8c1fe07e1f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:06.803227 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc52226-c57c-45ec-938a-9e177c44fddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:06.803825 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc52226-c57c-45ec-938a-9e177c44fddf admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:06.804106 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc52226-c57c-45ec-938a-9e177c44fddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:06.804379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc52226-c57c-45ec-938a-9e177c44fddf admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:06.805003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc52226-c57c-45ec-938a-9e177c44fddf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:06.806922 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc52226-c57c-45ec-938a-9e177c44fddf admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:06.881692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1419/2812] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:15:06 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:06.882862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f9a96-5be2-4980-baf7-2f8c0c7de9e7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:06.896003 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f9a96-5be2-4980-baf7-2f8c0c7de9e7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:06.954457 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c2022-fbfe-47bf-8c6a-0558e97573fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:06.955117 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c2022-fbfe-47bf-8c6a-0558e97573fe admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:06.955394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c2022-fbfe-47bf-8c6a-0558e97573fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:06.955666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c2022-fbfe-47bf-8c6a-0558e97573fe admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:06.956310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c2022-fbfe-47bf-8c6a-0558e97573fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:06.958250 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c2022-fbfe-47bf-8c6a-0558e97573fe admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:07.074867 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1394/2813] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:15:06 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:15:07.076315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc52226-c57c-45ec-938a-9e177c44fddf admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:07.086875 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc52226-c57c-45ec-938a-9e177c44fddf admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:07.091582 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf43a4b-b84c-47d3-afaa-4efb88ee545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:07.092179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf43a4b-b84c-47d3-afaa-4efb88ee545a admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:07.092460 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf43a4b-b84c-47d3-afaa-4efb88ee545a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:07.092732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf43a4b-b84c-47d3-afaa-4efb88ee545a admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:07.093374 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf43a4b-b84c-47d3-afaa-4efb88ee545a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:07.095149 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf43a4b-b84c-47d3-afaa-4efb88ee545a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:07.106123 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1420/2814] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:07.110224 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c2022-fbfe-47bf-8c6a-0558e97573fe admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:07.121686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c2022-fbfe-47bf-8c6a-0558e97573fe admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:07.127588 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ee47d-f82d-44cd-92c6-2e1bfa753a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:07.128252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ee47d-f82d-44cd-92c6-2e1bfa753a72 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:07.147707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ee47d-f82d-44cd-92c6-2e1bfa753a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:07.147707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ee47d-f82d-44cd-92c6-2e1bfa753a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:07.147707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ee47d-f82d-44cd-92c6-2e1bfa753a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:07.147707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ee47d-f82d-44cd-92c6-2e1bfa753a72 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:07.147707 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1395/2815] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:07.148392 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf43a4b-b84c-47d3-afaa-4efb88ee545a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:07.159689 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf43a4b-b84c-47d3-afaa-4efb88ee545a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:07.164977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c1dbd-b969-4229-b86a-5a421213340b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:07.165655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c1dbd-b969-4229-b86a-5a421213340b admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:07.165937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c1dbd-b969-4229-b86a-5a421213340b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:07.166215 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c1dbd-b969-4229-b86a-5a421213340b admin admin] PATH_INFO: `/v3/projects/c7e380ccc6ae49d5a7363e40dc7bb589/users/9790f35c96fd457381dc6e51508c8be8/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:07.184444 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c1dbd-b969-4229-b86a-5a421213340b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7e380ccc6ae49d5a7363e40dc7bb589, user_id=9790f35c96fd457381dc6e51508c8be8, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:07.187366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c1dbd-b969-4229-b86a-5a421213340b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:07.254853 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1421/2816] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:15:07 2022] PUT /identity/v3/projects/c7e380ccc6ae49d5a7363e40dc7bb589/users/9790f35c96fd457381dc6e51508c8be8/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:07.264568 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98630ec2-035c-46ae-b1ab-974c56c81aa0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:07.264860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98630ec2-035c-46ae-b1ab-974c56c81aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:07.265143 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98630ec2-035c-46ae-b1ab-974c56c81aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:07.302782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-98630ec2-035c-46ae-b1ab-974c56c81aa0 None None] MFA Rules not processed for user `9790f35c96fd457381dc6e51508c8be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:07.308750 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-98630ec2-035c-46ae-b1ab-974c56c81aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:07.324068 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:07.379569 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1396/2817] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:07.384235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c1dbd-b969-4229-b86a-5a421213340b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:07.448728 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c1dbd-b969-4229-b86a-5a421213340b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:07.453850 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca59ba-474f-49e9-af8e-4399768b74a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:07.454487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca59ba-474f-49e9-af8e-4399768b74a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:07.454801 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca59ba-474f-49e9-af8e-4399768b74a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:07.455078 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca59ba-474f-49e9-af8e-4399768b74a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:07.455711 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca59ba-474f-49e9-af8e-4399768b74a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:07.457719 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca59ba-474f-49e9-af8e-4399768b74a9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:07.529051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1422/2818] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:15:07 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:07.533795 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98630ec2-035c-46ae-b1ab-974c56c81aa0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:07.545046 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98630ec2-035c-46ae-b1ab-974c56c81aa0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:07.622978 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262cd3c-7238-4930-9adf-091633c0447d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:07.623632 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262cd3c-7238-4930-9adf-091633c0447d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:07.623992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262cd3c-7238-4930-9adf-091633c0447d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:07.624194 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262cd3c-7238-4930-9adf-091633c0447d admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:07.624839 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262cd3c-7238-4930-9adf-091633c0447d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:07.627006 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262cd3c-7238-4930-9adf-091633c0447d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:07.725968 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1397/2819] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:15:07 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:07.727315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca59ba-474f-49e9-af8e-4399768b74a9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:07.738040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca59ba-474f-49e9-af8e-4399768b74a9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:07.743071 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81771b80-014c-49cc-ab3a-ae3baaea53e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:07.743811 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81771b80-014c-49cc-ab3a-ae3baaea53e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:07.748155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81771b80-014c-49cc-ab3a-ae3baaea53e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:07.748155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81771b80-014c-49cc-ab3a-ae3baaea53e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:07.748155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81771b80-014c-49cc-ab3a-ae3baaea53e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:07.748155 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81771b80-014c-49cc-ab3a-ae3baaea53e5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:07.757509 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1423/2820] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:07.761763 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262cd3c-7238-4930-9adf-091633c0447d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:07.773591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262cd3c-7238-4930-9adf-091633c0447d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:07.778629 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bc30d-5134-481d-8d9d-384f9a6fbe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:07.779314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bc30d-5134-481d-8d9d-384f9a6fbe9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:07.779612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bc30d-5134-481d-8d9d-384f9a6fbe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:07.780026 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bc30d-5134-481d-8d9d-384f9a6fbe9e admin admin] PATH_INFO: `/v3/projects/a1bdd1867d8e4c4bba85b8e66f92df50/users/4b76ceb1d552455ab5b295ffffd2d032/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:07.795939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bc30d-5134-481d-8d9d-384f9a6fbe9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1bdd1867d8e4c4bba85b8e66f92df50, user_id=4b76ceb1d552455ab5b295ffffd2d032, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:07.798621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bc30d-5134-481d-8d9d-384f9a6fbe9e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:07.876555 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1398/2821] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:15:07 2022] PUT /identity/v3/projects/a1bdd1867d8e4c4bba85b8e66f92df50/users/4b76ceb1d552455ab5b295ffffd2d032/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:07.880421 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81771b80-014c-49cc-ab3a-ae3baaea53e5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:07.891111 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81771b80-014c-49cc-ab3a-ae3baaea53e5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:07.905505 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:07.958877 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b69d59-8576-49e8-8e92-db4f6930697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:07.959476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b69d59-8576-49e8-8e92-db4f6930697e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:07.959755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b69d59-8576-49e8-8e92-db4f6930697e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:07.960088 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b69d59-8576-49e8-8e92-db4f6930697e admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:07.960732 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b69d59-8576-49e8-8e92-db4f6930697e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:07.962321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b69d59-8576-49e8-8e92-db4f6930697e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:07.973401 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1424/2822] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:07.978065 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bc30d-5134-481d-8d9d-384f9a6fbe9e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:07.988675 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bc30d-5134-481d-8d9d-384f9a6fbe9e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:07.993299 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a745-87fd-46d4-bb06-0c691bfa2751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:07.993943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a745-87fd-46d4-bb06-0c691bfa2751 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:07.994220 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a745-87fd-46d4-bb06-0c691bfa2751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:07.994492 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a745-87fd-46d4-bb06-0c691bfa2751 admin admin] PATH_INFO: `/v3/projects/a1bdd1867d8e4c4bba85b8e66f92df50/users/4b76ceb1d552455ab5b295ffffd2d032/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:07.996499 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096a745-87fd-46d4-bb06-0c691bfa2751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1bdd1867d8e4c4bba85b8e66f92df50, user_id=4b76ceb1d552455ab5b295ffffd2d032, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:07.999190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096a745-87fd-46d4-bb06-0c691bfa2751 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:08.011518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1399/2823] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:15:07 2022] PUT /identity/v3/projects/a1bdd1867d8e4c4bba85b8e66f92df50/users/4b76ceb1d552455ab5b295ffffd2d032/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:08.022667 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c0ec8-3a90-4bf0-b604-d082d1e5f956 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:08.022987 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c0ec8-3a90-4bf0-b604-d082d1e5f956 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:08.023271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c0ec8-3a90-4bf0-b604-d082d1e5f956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:08.053336 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e66c0ec8-3a90-4bf0-b604-d082d1e5f956 None None] MFA Rules not processed for user `4b76ceb1d552455ab5b295ffffd2d032`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:08.054940 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e66c0ec8-3a90-4bf0-b604-d082d1e5f956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:08.137141 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1425/2824] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:08.149053 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a745-87fd-46d4-bb06-0c691bfa2751 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:08.150885 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8096a745-87fd-46d4-bb06-0c691bfa2751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:08.231182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a745-87fd-46d4-bb06-0c691bfa2751 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:08.231182 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c28b4b-80f3-4055-bc42-7aa457b7ae22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:08.231494 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c28b4b-80f3-4055-bc42-7aa457b7ae22 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:08.231801 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c28b4b-80f3-4055-bc42-7aa457b7ae22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:08.232164 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c28b4b-80f3-4055-bc42-7aa457b7ae22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:08.242628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c28b4b-80f3-4055-bc42-7aa457b7ae22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:08.244163 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c28b4b-80f3-4055-bc42-7aa457b7ae22 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:08.267823 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1400/2825] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:08.366413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c0ec8-3a90-4bf0-b604-d082d1e5f956 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:08.376068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c0ec8-3a90-4bf0-b604-d082d1e5f956 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:08.380261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a5b9d-8295-4b04-b34d-0dbf4400703b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:08.380949 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a5b9d-8295-4b04-b34d-0dbf4400703b service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:08.381235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a5b9d-8295-4b04-b34d-0dbf4400703b service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:08.381574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a5b9d-8295-4b04-b34d-0dbf4400703b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:08.448647 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a5b9d-8295-4b04-b34d-0dbf4400703b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:08.450407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a5b9d-8295-4b04-b34d-0dbf4400703b service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:08.468731 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1426/2826] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:12.394207 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c28b4b-80f3-4055-bc42-7aa457b7ae22 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:12.408333 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:12.458272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c28b4b-80f3-4055-bc42-7aa457b7ae22 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:12.464798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:12.465002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:12.465238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:12.465508 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] PATH_INFO: `/v3/users/5cb98dd23fe945b8b053b9d6e498a900` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:12.466743 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cb98dd23fe945b8b053b9d6e498a900)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:12.468603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:12.626114 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cb98dd23fe945b8b053b9d6e498a900'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:12.638354 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cb98dd23fe945b8b053b9d6e498a900'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:12.645980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cb98dd23fe945b8b053b9d6e498a900'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:12.721609 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cb98dd23fe945b8b053b9d6e498a900'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:12.722303 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cb98dd23fe945b8b053b9d6e498a900'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:12.728534 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1401/2827] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:15:12 2022] DELETE /identity/v3/users/5cb98dd23fe945b8b053b9d6e498a900 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:12.733150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e32d-da84-401d-98dc-18b85b8ceff6 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:12.733633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e32d-da84-401d-98dc-18b85b8ceff6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:12.733736 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e32d-da84-401d-98dc-18b85b8ceff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:12.753272 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ed06e32d-da84-401d-98dc-18b85b8ceff6 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:12.754514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ed06e32d-da84-401d-98dc-18b85b8ceff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:12.783436 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:12.812825 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1427/2828] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:15:12.823162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:12.824599 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:12.884943 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d79458-8177-4308-9de0-8d5c994bfd7a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:12.888528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98db683-f6f5-428c-ad78-048a20497a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:12.889130 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98db683-f6f5-428c-ad78-048a20497a67 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:12.889408 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98db683-f6f5-428c-ad78-048a20497a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:12.889680 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98db683-f6f5-428c-ad78-048a20497a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:12.898591 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98db683-f6f5-428c-ad78-048a20497a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:12.900042 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98db683-f6f5-428c-ad78-048a20497a67 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:12.909471 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1402/2829] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:13.284247 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06e32d-da84-401d-98dc-18b85b8ceff6 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:13.287550 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ed06e32d-da84-401d-98dc-18b85b8ceff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:13.298105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06e32d-da84-401d-98dc-18b85b8ceff6 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:13.303506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:13.304165 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:13.304455 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:13.304730 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] PATH_INFO: `/v3/projects/a8caae866029435bb715f06699a6da46` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:13.310385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8caae866029435bb715f06699a6da46)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:13.310385 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:13.443828 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8caae866029435bb715f06699a6da46', 'request_id': 'req-f2f7da30-2223-4348-8253-5afb670f2720'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:13.450351 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8caae866029435bb715f06699a6da46', 'request_id': 'req-f2f7da30-2223-4348-8253-5afb670f2720'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:13.456262 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8caae866029435bb715f06699a6da46', 'request_id': 'req-f2f7da30-2223-4348-8253-5afb670f2720'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:13.550587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] The token cache is being invalidate because project a8caae866029435bb715f06699a6da46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:15:13.550955 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:13.555852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:13.555852 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1428/2830] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:15:13 2022] DELETE /identity/v3/projects/a8caae866029435bb715f06699a6da46 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:13.558451 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98db683-f6f5-428c-ad78-048a20497a67 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:13.568956 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98db683-f6f5-428c-ad78-048a20497a67 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:13.600227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:13.625918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:13.626557 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:13.626872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:13.627178 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] PATH_INFO: `/v3/users/9790f35c96fd457381dc6e51508c8be8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:13.628310 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9790f35c96fd457381dc6e51508c8be8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:13.630240 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:13.814860 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9790f35c96fd457381dc6e51508c8be8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:13.827732 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9790f35c96fd457381dc6e51508c8be8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:13.835493 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9790f35c96fd457381dc6e51508c8be8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:13.910528 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9790f35c96fd457381dc6e51508c8be8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:13.911162 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9790f35c96fd457381dc6e51508c8be8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:13.917326 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1403/2831] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:15:13 2022] DELETE /identity/v3/users/9790f35c96fd457381dc6e51508c8be8 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:14.323971 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:14.327937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:14.338237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7da30-2223-4348-8253-5afb670f2720 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:14.397396 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:14.398034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:14.398313 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:14.398586 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] PATH_INFO: `/v3/projects/c7e380ccc6ae49d5a7363e40dc7bb589` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:14.399760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7e380ccc6ae49d5a7363e40dc7bb589)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:14.401978 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:14.521781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7e380ccc6ae49d5a7363e40dc7bb589', 'request_id': 'req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:14.528800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7e380ccc6ae49d5a7363e40dc7bb589', 'request_id': 'req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:14.532793 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:14.534967 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7e380ccc6ae49d5a7363e40dc7bb589', 'request_id': 'req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:14.595592 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] The token cache is being invalidate because project c7e380ccc6ae49d5a7363e40dc7bb589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:15:14.595815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:14.596432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:14.597933 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1429/2832] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:15:14 2022] DELETE /identity/v3/projects/c7e380ccc6ae49d5a7363e40dc7bb589 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:14.602150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:14.603577 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:14.616621 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe45158-65bd-406c-83d5-5369c31702b7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:14.672258 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a173010-8019-4123-b570-42ae69643f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:14.672895 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a173010-8019-4123-b570-42ae69643f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:14.673174 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a173010-8019-4123-b570-42ae69643f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:14.673449 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a173010-8019-4123-b570-42ae69643f5f admin admin] PATH_INFO: `/v3/users/4b76ceb1d552455ab5b295ffffd2d032` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:14.674607 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a173010-8019-4123-b570-42ae69643f5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b76ceb1d552455ab5b295ffffd2d032)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:14.677235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a173010-8019-4123-b570-42ae69643f5f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:14.761636 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:14.763121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:14.820762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a7c96-ecf8-44aa-a3ed-b3ecb0800c4b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:14.824419 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ef22b-23a6-4c7d-b5c4-52626875e498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:14.825003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ef22b-23a6-4c7d-b5c4-52626875e498 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:14.825284 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ef22b-23a6-4c7d-b5c4-52626875e498 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:14.825557 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ef22b-23a6-4c7d-b5c4-52626875e498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:14.827299 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2b1ef22b-23a6-4c7d-b5c4-52626875e498 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:14.863371 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2a173010-8019-4123-b570-42ae69643f5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b76ceb1d552455ab5b295ffffd2d032'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:14.868538 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:14.875710 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2a173010-8019-4123-b570-42ae69643f5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b76ceb1d552455ab5b295ffffd2d032'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:14.883767 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2a173010-8019-4123-b570-42ae69643f5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b76ceb1d552455ab5b295ffffd2d032'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:14.899760 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ef22b-23a6-4c7d-b5c4-52626875e498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:14.901453 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ef22b-23a6-4c7d-b5c4-52626875e498 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:14.913019 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1430/2833] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:15:14.956437 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2a173010-8019-4123-b570-42ae69643f5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b76ceb1d552455ab5b295ffffd2d032'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:14.957088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2a173010-8019-4123-b570-42ae69643f5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b76ceb1d552455ab5b295ffffd2d032'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:14.963703 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1404/2834] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:15:14 2022] DELETE /identity/v3/users/4b76ceb1d552455ab5b295ffffd2d032 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:15.421304 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ef22b-23a6-4c7d-b5c4-52626875e498 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:15.423070 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2b1ef22b-23a6-4c7d-b5c4-52626875e498 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:15.439836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ef22b-23a6-4c7d-b5c4-52626875e498 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:15.496613 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:15.497241 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:15.497516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:15.497788 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] PATH_INFO: `/v3/projects/a1bdd1867d8e4c4bba85b8e66f92df50` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:15.499326 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1bdd1867d8e4c4bba85b8e66f92df50)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:15.501205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:15.573649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:15.587366 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1bdd1867d8e4c4bba85b8e66f92df50', 'request_id': 'req-b01d3b02-f285-4776-9e4f-7bf544096505'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:15.593690 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1bdd1867d8e4c4bba85b8e66f92df50', 'request_id': 'req-b01d3b02-f285-4776-9e4f-7bf544096505'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:15.599900 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1bdd1867d8e4c4bba85b8e66f92df50', 'request_id': 'req-b01d3b02-f285-4776-9e4f-7bf544096505'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:15.659190 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] The token cache is being invalidate because project a1bdd1867d8e4c4bba85b8e66f92df50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:15:15.659482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:15.660109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b01d3b02-f285-4776-9e4f-7bf544096505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:15.661588 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1431/2835] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:15:15 2022] DELETE /identity/v3/projects/a1bdd1867d8e4c4bba85b8e66f92df50 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:15.671242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917801d-0c5e-4096-85c3-9ec2d2d81da0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:15.671242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917801d-0c5e-4096-85c3-9ec2d2d81da0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:15.671242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917801d-0c5e-4096-85c3-9ec2d2d81da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:15.688978 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b917801d-0c5e-4096-85c3-9ec2d2d81da0 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:15.690290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b917801d-0c5e-4096-85c3-9ec2d2d81da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:15.750990 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1405/2836] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:15.760648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0708-1970-4efc-b31d-7ebf6ae47c03 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:15.760932 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0708-1970-4efc-b31d-7ebf6ae47c03 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:15.761206 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0708-1970-4efc-b31d-7ebf6ae47c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:15.781716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0d7d0708-1970-4efc-b31d-7ebf6ae47c03 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:15.783091 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0d7d0708-1970-4efc-b31d-7ebf6ae47c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:15.791930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1432/2837] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:15.796297 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917801d-0c5e-4096-85c3-9ec2d2d81da0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:15.806730 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917801d-0c5e-4096-85c3-9ec2d2d81da0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:15.811342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e9576-76d9-4db0-b9fb-9fd5040064ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:15.811944 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e9576-76d9-4db0-b9fb-9fd5040064ab admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:15.812225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e9576-76d9-4db0-b9fb-9fd5040064ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:15.812497 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e9576-76d9-4db0-b9fb-9fd5040064ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:15.813140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e9576-76d9-4db0-b9fb-9fd5040064ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:15:15.813520 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e9576-76d9-4db0-b9fb-9fd5040064ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:15.815160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e9576-76d9-4db0-b9fb-9fd5040064ab admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:15.827032 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1406/2838] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:15:15.832147 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d0708-1970-4efc-b31d-7ebf6ae47c03 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:15.841623 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d0708-1970-4efc-b31d-7ebf6ae47c03 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:15.847045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e5a63-afc0-4883-b4e6-6d34442cda62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:15.847648 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e5a63-afc0-4883-b4e6-6d34442cda62 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:15.847930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e5a63-afc0-4883-b4e6-6d34442cda62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:15.848205 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e5a63-afc0-4883-b4e6-6d34442cda62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:15.848845 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e5a63-afc0-4883-b4e6-6d34442cda62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:15.850785 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e5a63-afc0-4883-b4e6-6d34442cda62 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:15.941034 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1433/2839] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:15:15 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:15.945314 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e9576-76d9-4db0-b9fb-9fd5040064ab admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:15.955467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e9576-76d9-4db0-b9fb-9fd5040064ab admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:16.012612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a948c5-d908-4b57-8bec-5ebbf0961b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:16.013226 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a948c5-d908-4b57-8bec-5ebbf0961b20 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:16.013530 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a948c5-d908-4b57-8bec-5ebbf0961b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:16.013799 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a948c5-d908-4b57-8bec-5ebbf0961b20 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:16.014422 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a948c5-d908-4b57-8bec-5ebbf0961b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:16.016377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a948c5-d908-4b57-8bec-5ebbf0961b20 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:16.092658 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1407/2840] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:15:15 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:16.093432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e5a63-afc0-4883-b4e6-6d34442cda62 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:16.104660 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e5a63-afc0-4883-b4e6-6d34442cda62 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:16.110150 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c6f9b-f381-4e5e-969c-cbadb112ecdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:16.112317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6f9b-f381-4e5e-969c-cbadb112ecdc admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:16.112317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6f9b-f381-4e5e-969c-cbadb112ecdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:16.112317 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6f9b-f381-4e5e-969c-cbadb112ecdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:16.112624 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c6f9b-f381-4e5e-969c-cbadb112ecdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:16.114207 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c6f9b-f381-4e5e-969c-cbadb112ecdc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:16.124983 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1434/2841] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:16.129798 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a948c5-d908-4b57-8bec-5ebbf0961b20 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:16.140124 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a948c5-d908-4b57-8bec-5ebbf0961b20 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:16.145157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f33df-7fcb-4ec5-a946-e340d83cde94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:16.145702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f33df-7fcb-4ec5-a946-e340d83cde94 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:16.145992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f33df-7fcb-4ec5-a946-e340d83cde94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:16.146271 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f33df-7fcb-4ec5-a946-e340d83cde94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:16.146962 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f33df-7fcb-4ec5-a946-e340d83cde94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:16.148765 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f33df-7fcb-4ec5-a946-e340d83cde94 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:16.166288 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1408/2842] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:16.169681 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c6f9b-f381-4e5e-969c-cbadb112ecdc admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:16.178457 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c6f9b-f381-4e5e-969c-cbadb112ecdc admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:16.183188 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494cd1a3-a275-4f49-97e9-0150f1a36fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:16.183851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494cd1a3-a275-4f49-97e9-0150f1a36fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:16.184135 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494cd1a3-a275-4f49-97e9-0150f1a36fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:16.184411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494cd1a3-a275-4f49-97e9-0150f1a36fe4 admin admin] PATH_INFO: `/v3/projects/69876b8ba5ff43038b9b71c16089ca1c/users/ec8014d6bbb14eed9ff2a2fce6d7ab02/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:16.201800 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494cd1a3-a275-4f49-97e9-0150f1a36fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69876b8ba5ff43038b9b71c16089ca1c, user_id=ec8014d6bbb14eed9ff2a2fce6d7ab02, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:16.204473 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494cd1a3-a275-4f49-97e9-0150f1a36fe4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:16.322343 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1435/2843] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:15:16 2022] PUT /identity/v3/projects/69876b8ba5ff43038b9b71c16089ca1c/users/ec8014d6bbb14eed9ff2a2fce6d7ab02/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 08:15:16.333579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58c321-4987-4789-b3f6-be396e25ebe1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:16.333870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58c321-4987-4789-b3f6-be396e25ebe1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:16.334153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58c321-4987-4789-b3f6-be396e25ebe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:16.363872 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fe58c321-4987-4789-b3f6-be396e25ebe1 None None] MFA Rules not processed for user `ec8014d6bbb14eed9ff2a2fce6d7ab02`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:16.365089 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fe58c321-4987-4789-b3f6-be396e25ebe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:16.390564 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:16.430027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1409/2844] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:16.434744 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494cd1a3-a275-4f49-97e9-0150f1a36fe4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:16.496486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494cd1a3-a275-4f49-97e9-0150f1a36fe4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:16.501235 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943c2bd-b7e1-4a3c-8d80-30535dbc9c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:16.501832 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943c2bd-b7e1-4a3c-8d80-30535dbc9c10 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:16.502134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943c2bd-b7e1-4a3c-8d80-30535dbc9c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:16.502408 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943c2bd-b7e1-4a3c-8d80-30535dbc9c10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:16.503164 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943c2bd-b7e1-4a3c-8d80-30535dbc9c10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:16.505040 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943c2bd-b7e1-4a3c-8d80-30535dbc9c10 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:16.573838 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1436/2845] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:15:16 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 08:15:16.577959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58c321-4987-4789-b3f6-be396e25ebe1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:16.589150 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58c321-4987-4789-b3f6-be396e25ebe1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:16.645040 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3723b0-9dc7-4146-9254-b2b68dce77d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:16.645643 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3723b0-9dc7-4146-9254-b2b68dce77d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:16.645924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3723b0-9dc7-4146-9254-b2b68dce77d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:16.646228 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3723b0-9dc7-4146-9254-b2b68dce77d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:16.646890 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3723b0-9dc7-4146-9254-b2b68dce77d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:16.648809 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3723b0-9dc7-4146-9254-b2b68dce77d9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:16.716179 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1410/2846] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:15:16 2022] POST /identity/v3/users => generated 288 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:16.720307 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943c2bd-b7e1-4a3c-8d80-30535dbc9c10 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:16.731637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943c2bd-b7e1-4a3c-8d80-30535dbc9c10 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:16.736315 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda8beb-9839-432d-a45d-680e71cfd217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:16.736930 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda8beb-9839-432d-a45d-680e71cfd217 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:16.737211 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda8beb-9839-432d-a45d-680e71cfd217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:16.737486 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda8beb-9839-432d-a45d-680e71cfd217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:16.738133 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda8beb-9839-432d-a45d-680e71cfd217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:16.739762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda8beb-9839-432d-a45d-680e71cfd217 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:16.750635 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1437/2847] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:16.754794 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3723b0-9dc7-4146-9254-b2b68dce77d9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:16.764958 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3723b0-9dc7-4146-9254-b2b68dce77d9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:16.770121 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f50f8-19a1-4f5a-8c62-7d0a4f19220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:16.770738 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f50f8-19a1-4f5a-8c62-7d0a4f19220d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:16.771019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f50f8-19a1-4f5a-8c62-7d0a4f19220d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:16.771291 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f50f8-19a1-4f5a-8c62-7d0a4f19220d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:16.771927 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f50f8-19a1-4f5a-8c62-7d0a4f19220d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:16.773514 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f50f8-19a1-4f5a-8c62-7d0a4f19220d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:16.784082 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1411/2848] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:16.787977 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda8beb-9839-432d-a45d-680e71cfd217 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:16.797890 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda8beb-9839-432d-a45d-680e71cfd217 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:16.802658 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc36e7d-5834-441c-af44-66c741611578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:16.803331 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc36e7d-5834-441c-af44-66c741611578 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:16.803620 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc36e7d-5834-441c-af44-66c741611578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:16.803896 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc36e7d-5834-441c-af44-66c741611578 admin admin] PATH_INFO: `/v3/projects/4eea1c2acdbc4cd3b34ab75b433b34c1/users/1dda7a1f3b8047369901707ec1e42fbb/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:16.818804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc36e7d-5834-441c-af44-66c741611578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eea1c2acdbc4cd3b34ab75b433b34c1, user_id=1dda7a1f3b8047369901707ec1e42fbb, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:16.821489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc36e7d-5834-441c-af44-66c741611578 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:16.887388 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1438/2849] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:15:16 2022] PUT /identity/v3/projects/4eea1c2acdbc4cd3b34ab75b433b34c1/users/1dda7a1f3b8047369901707ec1e42fbb/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:16.899113 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1f38b-f06e-4c19-a890-f89be430a0e2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:16.899405 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1f38b-f06e-4c19-a890-f89be430a0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:16.899705 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1f38b-f06e-4c19-a890-f89be430a0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:16.928673 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-afa1f38b-f06e-4c19-a890-f89be430a0e2 None None] MFA Rules not processed for user `1dda7a1f3b8047369901707ec1e42fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:16.929889 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-afa1f38b-f06e-4c19-a890-f89be430a0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:16.996843 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1412/2850] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:17.001432 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc36e7d-5834-441c-af44-66c741611578 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:17.007733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:17.067597 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc36e7d-5834-441c-af44-66c741611578 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:17.072173 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914d826-030b-4281-905d-017d826484ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:17.072844 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d826-030b-4281-905d-017d826484ae admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:17.073059 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d826-030b-4281-905d-017d826484ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:17.073333 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d826-030b-4281-905d-017d826484ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:17.073958 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914d826-030b-4281-905d-017d826484ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:17.075880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914d826-030b-4281-905d-017d826484ae admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:17.183414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1439/2851] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:15:17 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:17.188776 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1f38b-f06e-4c19-a890-f89be430a0e2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:17.196560 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1f38b-f06e-4c19-a890-f89be430a0e2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:17.230832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:17.257282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ac70d-8337-47f3-95c4-9061af10d163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:17.257282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ac70d-8337-47f3-95c4-9061af10d163 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:17.257282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ac70d-8337-47f3-95c4-9061af10d163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:17.257282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ac70d-8337-47f3-95c4-9061af10d163 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:17.257793 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ac70d-8337-47f3-95c4-9061af10d163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:17.259763 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ac70d-8337-47f3-95c4-9061af10d163 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:17.362814 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1413/2852] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:15:17 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:17.366856 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914d826-030b-4281-905d-017d826484ae admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:17.378188 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914d826-030b-4281-905d-017d826484ae admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:17.383786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0c272-b539-4c60-8746-c09e8159e8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:17.383954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0c272-b539-4c60-8746-c09e8159e8bf admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:17.384236 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0c272-b539-4c60-8746-c09e8159e8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:17.384515 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0c272-b539-4c60-8746-c09e8159e8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:17.385166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0c272-b539-4c60-8746-c09e8159e8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:17.386865 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0c272-b539-4c60-8746-c09e8159e8bf admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:17.401379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1440/2853] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 08:15:17.405204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ac70d-8337-47f3-95c4-9061af10d163 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:17.416830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ac70d-8337-47f3-95c4-9061af10d163 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:17.423803 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2ade6-92dc-4778-8bf6-787d1d61c275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:17.425112 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ade6-92dc-4778-8bf6-787d1d61c275 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:17.425361 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ade6-92dc-4778-8bf6-787d1d61c275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:17.425715 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ade6-92dc-4778-8bf6-787d1d61c275 admin admin] PATH_INFO: `/v3/projects/0f2b73cad8c7429382b0772521a8edfd/users/c8f9bf239de3451897a8636e5bdc9bc7/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:17.445650 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2ade6-92dc-4778-8bf6-787d1d61c275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2b73cad8c7429382b0772521a8edfd, user_id=c8f9bf239de3451897a8636e5bdc9bc7, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:17.448340 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2ade6-92dc-4778-8bf6-787d1d61c275 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:17.517386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1414/2854] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:15:17 2022] PUT /identity/v3/projects/0f2b73cad8c7429382b0772521a8edfd/users/c8f9bf239de3451897a8636e5bdc9bc7/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:17.521482 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0c272-b539-4c60-8746-c09e8159e8bf admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:17.531643 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0c272-b539-4c60-8746-c09e8159e8bf admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:17.587329 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305ce91-f3a7-419f-ab53-9eeb85ce274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:17.588887 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305ce91-f3a7-419f-ab53-9eeb85ce274e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:17.589209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305ce91-f3a7-419f-ab53-9eeb85ce274e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:17.589480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305ce91-f3a7-419f-ab53-9eeb85ce274e admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:17.590118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305ce91-f3a7-419f-ab53-9eeb85ce274e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:17.591715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305ce91-f3a7-419f-ab53-9eeb85ce274e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:17.601379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1441/2855] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:17.605825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2ade6-92dc-4778-8bf6-787d1d61c275 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:17.617088 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2ade6-92dc-4778-8bf6-787d1d61c275 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:17.623342 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212fca7-3bb2-4669-a745-edb332641882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:17.624013 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212fca7-3bb2-4669-a745-edb332641882 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:17.624294 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212fca7-3bb2-4669-a745-edb332641882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:17.624562 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212fca7-3bb2-4669-a745-edb332641882 admin admin] PATH_INFO: `/v3/projects/0f2b73cad8c7429382b0772521a8edfd/users/c8f9bf239de3451897a8636e5bdc9bc7/roles/62eb3e525077429aacc0ada63300802e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:17.626491 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212fca7-3bb2-4669-a745-edb332641882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2b73cad8c7429382b0772521a8edfd, user_id=c8f9bf239de3451897a8636e5bdc9bc7, role_id=62eb3e525077429aacc0ada63300802e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:17.629828 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212fca7-3bb2-4669-a745-edb332641882 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:17.640412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1415/2856] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:15:17 2022] PUT /identity/v3/projects/0f2b73cad8c7429382b0772521a8edfd/users/c8f9bf239de3451897a8636e5bdc9bc7/roles/62eb3e525077429aacc0ada63300802e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:17.651570 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1fb46-8363-468d-9d25-5989f4b6f48c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:17.651857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1fb46-8363-468d-9d25-5989f4b6f48c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:17.652134 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1fb46-8363-468d-9d25-5989f4b6f48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:17.680897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7be1fb46-8363-468d-9d25-5989f4b6f48c None None] MFA Rules not processed for user `c8f9bf239de3451897a8636e5bdc9bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:17.682628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7be1fb46-8363-468d-9d25-5989f4b6f48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:17.746174 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:17.749718 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1442/2857] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:17.761914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212fca7-3bb2-4669-a745-edb332641882 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:17.763701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2212fca7-3bb2-4669-a745-edb332641882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:17.822746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212fca7-3bb2-4669-a745-edb332641882 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:17.826290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a34db5-e029-4e68-8afb-2a0bdf1706ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:17.826926 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a34db5-e029-4e68-8afb-2a0bdf1706ae service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:17.827209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a34db5-e029-4e68-8afb-2a0bdf1706ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:17.827488 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a34db5-e029-4e68-8afb-2a0bdf1706ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:17.884261 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a34db5-e029-4e68-8afb-2a0bdf1706ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:17.885702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a34db5-e029-4e68-8afb-2a0bdf1706ae service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:17.894746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1416/2858] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:18.512335 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1fb46-8363-468d-9d25-5989f4b6f48c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:18.513714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7be1fb46-8363-468d-9d25-5989f4b6f48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:18.589716 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1fb46-8363-468d-9d25-5989f4b6f48c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:18.593836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9c3d5-d0b8-49d3-9ea6-0b93d291bc9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:18.594496 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9c3d5-d0b8-49d3-9ea6-0b93d291bc9e service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:18.594934 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9c3d5-d0b8-49d3-9ea6-0b93d291bc9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:18.595326 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9c3d5-d0b8-49d3-9ea6-0b93d291bc9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:18.597166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f6e9c3d5-d0b8-49d3-9ea6-0b93d291bc9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:18.672788 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9c3d5-d0b8-49d3-9ea6-0b93d291bc9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:18.674261 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9c3d5-d0b8-49d3-9ea6-0b93d291bc9e service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:18.685411 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1443/2859] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:19.794022 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a34db5-e029-4e68-8afb-2a0bdf1706ae service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:19.803373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a34db5-e029-4e68-8afb-2a0bdf1706ae service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:19.807122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f68f94-ca5e-4aa5-a53d-83a43957ba6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:19.807824 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f68f94-ca5e-4aa5-a53d-83a43957ba6f service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:19.808107 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f68f94-ca5e-4aa5-a53d-83a43957ba6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:19.808377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f68f94-ca5e-4aa5-a53d-83a43957ba6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:19.853161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:19.869678 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f68f94-ca5e-4aa5-a53d-83a43957ba6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:19.871206 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f68f94-ca5e-4aa5-a53d-83a43957ba6f service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:19.881505 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1417/2860] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:21.490099 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9c3d5-d0b8-49d3-9ea6-0b93d291bc9e service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:21.499709 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9c3d5-d0b8-49d3-9ea6-0b93d291bc9e service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:21.503251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037267bb-d7eb-41dc-a346-d5c024aaf62a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:21.503876 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037267bb-d7eb-41dc-a346-d5c024aaf62a service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:21.504157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037267bb-d7eb-41dc-a346-d5c024aaf62a service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:21.504430 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037267bb-d7eb-41dc-a346-d5c024aaf62a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:21.513751 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037267bb-d7eb-41dc-a346-d5c024aaf62a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:21.515201 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037267bb-d7eb-41dc-a346-d5c024aaf62a service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:21.524531 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1444/2861] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:24.876123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f68f94-ca5e-4aa5-a53d-83a43957ba6f service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:24.876123 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d1f68f94-ca5e-4aa5-a53d-83a43957ba6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:24.881188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f68f94-ca5e-4aa5-a53d-83a43957ba6f service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:24.885137 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d1542-f0d0-4671-b5c5-54fed4093cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:24.885841 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d1542-f0d0-4671-b5c5-54fed4093cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:24.886221 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d1542-f0d0-4671-b5c5-54fed4093cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:24.886625 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d1542-f0d0-4671-b5c5-54fed4093cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:24.888757 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f31d1542-f0d0-4671-b5c5-54fed4093cc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:24.957245 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d1542-f0d0-4671-b5c5-54fed4093cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:24.958713 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d1542-f0d0-4671-b5c5-54fed4093cc9 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:24.968924 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1418/2862] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:28.398309 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037267bb-d7eb-41dc-a346-d5c024aaf62a service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:28.459880 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037267bb-d7eb-41dc-a346-d5c024aaf62a service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:28.464398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:28.465018 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:28.465297 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:28.465572 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] PATH_INFO: `/v3/users/ec8014d6bbb14eed9ff2a2fce6d7ab02` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:28.466669 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec8014d6bbb14eed9ff2a2fce6d7ab02)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:28.468608 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:28.585012 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:28.598178 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec8014d6bbb14eed9ff2a2fce6d7ab02'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:28.609812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec8014d6bbb14eed9ff2a2fce6d7ab02'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:28.616652 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec8014d6bbb14eed9ff2a2fce6d7ab02'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:28.652780 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec8014d6bbb14eed9ff2a2fce6d7ab02'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:28.653389 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec8014d6bbb14eed9ff2a2fce6d7ab02'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:28.659928 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1445/2863] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:15:28 2022] DELETE /identity/v3/users/ec8014d6bbb14eed9ff2a2fce6d7ab02 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:28.664635 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843cf10-1ef4-4a67-b6fb-01e19733f164 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:28.667237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843cf10-1ef4-4a67-b6fb-01e19733f164 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:28.667237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843cf10-1ef4-4a67-b6fb-01e19733f164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:28.683894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e843cf10-1ef4-4a67-b6fb-01e19733f164 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:28.685094 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e843cf10-1ef4-4a67-b6fb-01e19733f164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:28.744908 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1419/2864] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:28.754704 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:28.756198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:28.816813 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaeffa-9442-4786-a71e-541e0358f533 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:28.820466 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54364b37-c0ac-4424-8951-a78fec666c6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:28.821068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54364b37-c0ac-4424-8951-a78fec666c6f service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:28.821344 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54364b37-c0ac-4424-8951-a78fec666c6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:28.821614 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54364b37-c0ac-4424-8951-a78fec666c6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:28.830816 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54364b37-c0ac-4424-8951-a78fec666c6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:28.832747 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54364b37-c0ac-4424-8951-a78fec666c6f service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:28.842076 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1446/2865] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:29.255148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843cf10-1ef4-4a67-b6fb-01e19733f164 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:29.256628 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e843cf10-1ef4-4a67-b6fb-01e19733f164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:29.271002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843cf10-1ef4-4a67-b6fb-01e19733f164 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:29.275728 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:29.276620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:29.276898 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:29.277165 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] PATH_INFO: `/v3/projects/69876b8ba5ff43038b9b71c16089ca1c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:29.278316 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69876b8ba5ff43038b9b71c16089ca1c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:29.280268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:29.397135 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69876b8ba5ff43038b9b71c16089ca1c', 'request_id': 'req-6deca808-f2ba-497f-a0db-f7934a4d5ac6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:29.403777 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69876b8ba5ff43038b9b71c16089ca1c', 'request_id': 'req-6deca808-f2ba-497f-a0db-f7934a4d5ac6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:29.409308 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69876b8ba5ff43038b9b71c16089ca1c', 'request_id': 'req-6deca808-f2ba-497f-a0db-f7934a4d5ac6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:29.488737 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:29.493615 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] The token cache is being invalidate because project 69876b8ba5ff43038b9b71c16089ca1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:15:29.493923 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:29.494524 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:29.496050 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1420/2866] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:15:29 2022] DELETE /identity/v3/projects/69876b8ba5ff43038b9b71c16089ca1c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:29.500184 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54364b37-c0ac-4424-8951-a78fec666c6f service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:29.510571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54364b37-c0ac-4424-8951-a78fec666c6f service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:29.569409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:29.570048 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:29.570328 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:29.570600 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] PATH_INFO: `/v3/users/1dda7a1f3b8047369901707ec1e42fbb` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:29.571784 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dda7a1f3b8047369901707ec1e42fbb)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:29.573738 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:29.747781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dda7a1f3b8047369901707ec1e42fbb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:29.752289 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:29.760109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dda7a1f3b8047369901707ec1e42fbb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:29.766836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dda7a1f3b8047369901707ec1e42fbb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:29.818924 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dda7a1f3b8047369901707ec1e42fbb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:29.819745 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dda7a1f3b8047369901707ec1e42fbb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:29.825970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1447/2867] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:15:29 2022] DELETE /identity/v3/users/1dda7a1f3b8047369901707ec1e42fbb => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:30.257190 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:30.258988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:30.272667 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deca808-f2ba-497f-a0db-f7934a4d5ac6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:30.331700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:30.332827 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:30.333106 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:30.333377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] PATH_INFO: `/v3/projects/4eea1c2acdbc4cd3b34ab75b433b34c1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:30.334488 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eea1c2acdbc4cd3b34ab75b433b34c1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:30.336533 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:30.427290 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eea1c2acdbc4cd3b34ab75b433b34c1', 'request_id': 'req-bb54b29a-9531-4a04-8db0-d7469d037292'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:30.436353 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eea1c2acdbc4cd3b34ab75b433b34c1', 'request_id': 'req-bb54b29a-9531-4a04-8db0-d7469d037292'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:30.442952 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eea1c2acdbc4cd3b34ab75b433b34c1', 'request_id': 'req-bb54b29a-9531-4a04-8db0-d7469d037292'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:30.492401 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] The token cache is being invalidate because project 4eea1c2acdbc4cd3b34ab75b433b34c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:15:30.497435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:30.497435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:30.497435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1421/2868] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:15:30 2022] DELETE /identity/v3/projects/4eea1c2acdbc4cd3b34ab75b433b34c1 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:30.499095 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:30.501000 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:30.514415 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35605c4-7ff1-4cd3-9892-a52f4328ca08 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:30.574584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86df16-375d-4b98-89c6-2544b4148f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:30.575416 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86df16-375d-4b98-89c6-2544b4148f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:30.575699 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86df16-375d-4b98-89c6-2544b4148f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:30.575972 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86df16-375d-4b98-89c6-2544b4148f93 admin admin] PATH_INFO: `/v3/users/c8f9bf239de3451897a8636e5bdc9bc7` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:30.577144 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86df16-375d-4b98-89c6-2544b4148f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f9bf239de3451897a8636e5bdc9bc7)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:30.579157 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86df16-375d-4b98-89c6-2544b4148f93 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:30.719413 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1c86df16-375d-4b98-89c6-2544b4148f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f9bf239de3451897a8636e5bdc9bc7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:30.733442 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1c86df16-375d-4b98-89c6-2544b4148f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f9bf239de3451897a8636e5bdc9bc7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:30.741547 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1c86df16-375d-4b98-89c6-2544b4148f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f9bf239de3451897a8636e5bdc9bc7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:30.796242 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1c86df16-375d-4b98-89c6-2544b4148f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f9bf239de3451897a8636e5bdc9bc7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:30.796857 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1c86df16-375d-4b98-89c6-2544b4148f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f9bf239de3451897a8636e5bdc9bc7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:30.804264 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1448/2869] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:15:30 2022] DELETE /identity/v3/users/c8f9bf239de3451897a8636e5bdc9bc7 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:31.208574 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:31.210016 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:31.223665 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54b29a-9531-4a04-8db0-d7469d037292 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:31.285909 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:31.286550 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:31.286866 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:31.287139 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] PATH_INFO: `/v3/projects/0f2b73cad8c7429382b0772521a8edfd` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:31.288294 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f2b73cad8c7429382b0772521a8edfd)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:31.290378 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:31.378376 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:31.390811 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f2b73cad8c7429382b0772521a8edfd', 'request_id': 'req-cb748229-6b30-4e41-a65f-732eee33cbd9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:31.397237 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f2b73cad8c7429382b0772521a8edfd', 'request_id': 'req-cb748229-6b30-4e41-a65f-732eee33cbd9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:31.402995 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f2b73cad8c7429382b0772521a8edfd', 'request_id': 'req-cb748229-6b30-4e41-a65f-732eee33cbd9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:31.487602 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] The token cache is being invalidate because project 0f2b73cad8c7429382b0772521a8edfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:15:31.490781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:31.490781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cb748229-6b30-4e41-a65f-732eee33cbd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:31.490781 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1422/2870] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:15:31 2022] DELETE /identity/v3/projects/0f2b73cad8c7429382b0772521a8edfd => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:31.497651 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7143d-107c-4868-b4b5-7da19d307b04 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:31.497948 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7143d-107c-4868-b4b5-7da19d307b04 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:31.498231 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7143d-107c-4868-b4b5-7da19d307b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:31.519279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d8a7143d-107c-4868-b4b5-7da19d307b04 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:31.519279 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d8a7143d-107c-4868-b4b5-7da19d307b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:31.564607 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:31.579424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1449/2871] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:31.587961 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac04388-5e0d-460f-8c94-1ad213e93630 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:31.588197 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac04388-5e0d-460f-8c94-1ad213e93630 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:31.588468 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac04388-5e0d-460f-8c94-1ad213e93630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:31.608830 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1ac04388-5e0d-460f-8c94-1ad213e93630 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:31.610034 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1ac04388-5e0d-460f-8c94-1ad213e93630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:31.617918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1423/2872] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:31.628729 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7143d-107c-4868-b4b5-7da19d307b04 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:31.638303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7143d-107c-4868-b4b5-7da19d307b04 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:31.642941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7f5b3-9e8b-458c-949a-58ea1c1a8fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:31.643536 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7f5b3-9e8b-458c-949a-58ea1c1a8fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:31.643812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7f5b3-9e8b-458c-949a-58ea1c1a8fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:31.644080 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7f5b3-9e8b-458c-949a-58ea1c1a8fb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:31.645243 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7f5b3-9e8b-458c-949a-58ea1c1a8fb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:15:31.645542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7f5b3-9e8b-458c-949a-58ea1c1a8fb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:31.647239 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7f5b3-9e8b-458c-949a-58ea1c1a8fb8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:31.658863 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1450/2873] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:15:31.663988 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac04388-5e0d-460f-8c94-1ad213e93630 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:31.673146 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac04388-5e0d-460f-8c94-1ad213e93630 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:31.677746 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c11c1b-4098-4a13-bf52-d0416b050e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:31.679957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c11c1b-4098-4a13-bf52-d0416b050e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:31.679957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c11c1b-4098-4a13-bf52-d0416b050e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:31.679957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c11c1b-4098-4a13-bf52-d0416b050e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:31.679957 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c11c1b-4098-4a13-bf52-d0416b050e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:31.681382 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c11c1b-4098-4a13-bf52-d0416b050e6f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:31.781618 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1424/2874] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:15:31 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:31.785845 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7f5b3-9e8b-458c-949a-58ea1c1a8fb8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:31.795786 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7f5b3-9e8b-458c-949a-58ea1c1a8fb8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:31.853919 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75bf74-097f-4fba-87bd-09e58b514642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:31.854528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bf74-097f-4fba-87bd-09e58b514642 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:31.854841 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bf74-097f-4fba-87bd-09e58b514642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:31.855229 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75bf74-097f-4fba-87bd-09e58b514642 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:31.855882 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75bf74-097f-4fba-87bd-09e58b514642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:31.857833 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75bf74-097f-4fba-87bd-09e58b514642 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:31.927086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1451/2875] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:15:31 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:31.931448 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c11c1b-4098-4a13-bf52-d0416b050e6f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:31.942613 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c11c1b-4098-4a13-bf52-d0416b050e6f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:31.947459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a3a1f-435e-4b4f-8710-f555d78c345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:31.948054 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3a1f-435e-4b4f-8710-f555d78c345e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:31.948346 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3a1f-435e-4b4f-8710-f555d78c345e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:31.948666 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3a1f-435e-4b4f-8710-f555d78c345e admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:31.949307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a3a1f-435e-4b4f-8710-f555d78c345e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:31.950936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a3a1f-435e-4b4f-8710-f555d78c345e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:31.963571 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1425/2876] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:31.967367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75bf74-097f-4fba-87bd-09e58b514642 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:31.977438 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75bf74-097f-4fba-87bd-09e58b514642 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:31.981978 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae44b38-dd91-4ee6-b18e-dd32c5fb4127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:31.982565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae44b38-dd91-4ee6-b18e-dd32c5fb4127 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:31.982872 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae44b38-dd91-4ee6-b18e-dd32c5fb4127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:31.983146 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae44b38-dd91-4ee6-b18e-dd32c5fb4127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:31.983773 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae44b38-dd91-4ee6-b18e-dd32c5fb4127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:31.985355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae44b38-dd91-4ee6-b18e-dd32c5fb4127 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:31.996353 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1452/2877] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:32.000447 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a3a1f-435e-4b4f-8710-f555d78c345e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:32.010959 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a3a1f-435e-4b4f-8710-f555d78c345e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:32.015518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49294c3-373d-4233-9ebe-470ef0a97c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:32.016158 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49294c3-373d-4233-9ebe-470ef0a97c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:32.016435 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49294c3-373d-4233-9ebe-470ef0a97c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:32.016721 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49294c3-373d-4233-9ebe-470ef0a97c48 admin admin] PATH_INFO: `/v3/projects/189501ce59cd40bcbdf98964e256838f/users/012986457f7f42b295a3bca580bb7303/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:32.038816 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49294c3-373d-4233-9ebe-470ef0a97c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=189501ce59cd40bcbdf98964e256838f, user_id=012986457f7f42b295a3bca580bb7303, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:32.038816 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49294c3-373d-4233-9ebe-470ef0a97c48 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:32.110331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1426/2878] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:15:31 2022] PUT /identity/v3/projects/189501ce59cd40bcbdf98964e256838f/users/012986457f7f42b295a3bca580bb7303/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:32.121489 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c5b10-a6c7-46e2-aca7-74814124a433 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:32.121771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c5b10-a6c7-46e2-aca7-74814124a433 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:32.122045 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c5b10-a6c7-46e2-aca7-74814124a433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:32.150068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-242c5b10-a6c7-46e2-aca7-74814124a433 None None] MFA Rules not processed for user `012986457f7f42b295a3bca580bb7303`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:32.151555 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-242c5b10-a6c7-46e2-aca7-74814124a433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:32.213494 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1453/2879] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:15:32.225386 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49294c3-373d-4233-9ebe-470ef0a97c48 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:32.227453 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f49294c3-373d-4233-9ebe-470ef0a97c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:32.284672 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49294c3-373d-4233-9ebe-470ef0a97c48 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:32.288321 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92be0a-2468-4896-8adc-bfa89743a5f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:32.288979 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92be0a-2468-4896-8adc-bfa89743a5f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:32.289209 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92be0a-2468-4896-8adc-bfa89743a5f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:32.289483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92be0a-2468-4896-8adc-bfa89743a5f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:32.297825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92be0a-2468-4896-8adc-bfa89743a5f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:32.299302 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92be0a-2468-4896-8adc-bfa89743a5f2 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:32.308470 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1427/2880] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:33.995814 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90515362-cc03-4f51-8590-4f511151364a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:33.996381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90515362-cc03-4f51-8590-4f511151364a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:33.996506 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90515362-cc03-4f51-8590-4f511151364a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:34.017102 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-90515362-cc03-4f51-8590-4f511151364a None None] MFA Rules not processed for user `012986457f7f42b295a3bca580bb7303`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:34.018367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-90515362-cc03-4f51-8590-4f511151364a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:34.025548 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1454/2881] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:15:35.815033 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92be0a-2468-4896-8adc-bfa89743a5f2 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:35.816579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fd92be0a-2468-4896-8adc-bfa89743a5f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:35.888631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:35.892992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92be0a-2468-4896-8adc-bfa89743a5f2 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:35.898043 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ccdf4-7869-46bc-8e4b-d656af733e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:35.898702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ccdf4-7869-46bc-8e4b-d656af733e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:35.899004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ccdf4-7869-46bc-8e4b-d656af733e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:35.899307 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ccdf4-7869-46bc-8e4b-d656af733e44 admin admin] PATH_INFO: `/v3/users/cc14169c8f7044c6815944af5bc658a9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:35.900469 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ccdf4-7869-46bc-8e4b-d656af733e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc14169c8f7044c6815944af5bc658a9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:35.902708 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ccdf4-7869-46bc-8e4b-d656af733e44 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:36.066603 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1d7ccdf4-7869-46bc-8e4b-d656af733e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc14169c8f7044c6815944af5bc658a9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:36.078295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1d7ccdf4-7869-46bc-8e4b-d656af733e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc14169c8f7044c6815944af5bc658a9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:36.085681 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1d7ccdf4-7869-46bc-8e4b-d656af733e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc14169c8f7044c6815944af5bc658a9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:36.162644 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1d7ccdf4-7869-46bc-8e4b-d656af733e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc14169c8f7044c6815944af5bc658a9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:36.163396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1d7ccdf4-7869-46bc-8e4b-d656af733e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc14169c8f7044c6815944af5bc658a9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:36.169821 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1428/2882] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:15:35 2022] DELETE /identity/v3/users/cc14169c8f7044c6815944af5bc658a9 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:36.677077 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90515362-cc03-4f51-8590-4f511151364a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:36.688471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90515362-cc03-4f51-8590-4f511151364a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:36.730516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:36.755619 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:36.756259 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:36.756537 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:36.756808 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] PATH_INFO: `/v3/projects/e3276981b48d4974841dd97ee1cb9882` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:36.757942 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3276981b48d4974841dd97ee1cb9882)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:36.759874 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:36.865332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3276981b48d4974841dd97ee1cb9882', 'request_id': 'req-590c25ac-3089-4b65-9453-720f178bae44'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:36.871767 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3276981b48d4974841dd97ee1cb9882', 'request_id': 'req-590c25ac-3089-4b65-9453-720f178bae44'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:36.878172 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3276981b48d4974841dd97ee1cb9882', 'request_id': 'req-590c25ac-3089-4b65-9453-720f178bae44'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:36.955571 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] The token cache is being invalidate because project e3276981b48d4974841dd97ee1cb9882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:15:36.955804 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:36.956383 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-590c25ac-3089-4b65-9453-720f178bae44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:36.957870 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1455/2883] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:15:36 2022] DELETE /identity/v3/projects/e3276981b48d4974841dd97ee1cb9882 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:36.962755 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e882d8-d05c-43ab-89c4-610421282455 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:36.963268 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e882d8-d05c-43ab-89c4-610421282455 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:36.963479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e882d8-d05c-43ab-89c4-610421282455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:36.984380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-28e882d8-d05c-43ab-89c4-610421282455 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:36.985686 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-28e882d8-d05c-43ab-89c4-610421282455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:37.049312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1429/2884] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:15:37.058073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aff2a4-ac7c-43ec-a9fc-edad85b26c8d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:37.058073 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aff2a4-ac7c-43ec-a9fc-edad85b26c8d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:37.058379 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aff2a4-ac7c-43ec-a9fc-edad85b26c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:37.081208 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c7aff2a4-ac7c-43ec-a9fc-edad85b26c8d None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:37.088583 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c7aff2a4-ac7c-43ec-a9fc-edad85b26c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:37.091836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1456/2885] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 08:15:37.095980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e882d8-d05c-43ab-89c4-610421282455 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:37.106122 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e882d8-d05c-43ab-89c4-610421282455 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:37.110846 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b094b-7dbf-4489-984c-c3806b44b18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:37.111442 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b094b-7dbf-4489-984c-c3806b44b18c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:37.111722 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b094b-7dbf-4489-984c-c3806b44b18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:37.111994 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b094b-7dbf-4489-984c-c3806b44b18c admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:37.112677 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b094b-7dbf-4489-984c-c3806b44b18c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 08:15:37.112980 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b094b-7dbf-4489-984c-c3806b44b18c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:37.114632 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b094b-7dbf-4489-984c-c3806b44b18c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:37.126300 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1430/2886] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Sep 16 08:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 08:15:37.131198 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aff2a4-ac7c-43ec-a9fc-edad85b26c8d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:37.140626 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aff2a4-ac7c-43ec-a9fc-edad85b26c8d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:37.146182 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1336a-cdf5-4d25-9a0e-607b972d6cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:37.146846 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1336a-cdf5-4d25-9a0e-607b972d6cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:37.147233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1336a-cdf5-4d25-9a0e-607b972d6cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:37.147507 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1336a-cdf5-4d25-9a0e-607b972d6cf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:37.148168 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1336a-cdf5-4d25-9a0e-607b972d6cf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:37.150086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1336a-cdf5-4d25-9a0e-607b972d6cf6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:37.241639 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1457/2887] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Sep 16 08:15:37 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:37.242702 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b094b-7dbf-4489-984c-c3806b44b18c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:37.252826 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b094b-7dbf-4489-984c-c3806b44b18c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:37.309105 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a6be9-eb42-4b10-b1d5-f90a381993b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:37.309701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a6be9-eb42-4b10-b1d5-f90a381993b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:37.309979 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a6be9-eb42-4b10-b1d5-f90a381993b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:37.310252 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a6be9-eb42-4b10-b1d5-f90a381993b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:37.310900 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a6be9-eb42-4b10-b1d5-f90a381993b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:37.312825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a6be9-eb42-4b10-b1d5-f90a381993b0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:37.322887 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:37.439745 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1431/2888] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Sep 16 08:15:37 2022] POST /identity/v3/users => generated 300 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 08:15:37.441104 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1336a-cdf5-4d25-9a0e-607b972d6cf6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:37.451443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1336a-cdf5-4d25-9a0e-607b972d6cf6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:37.456439 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23dd9d-9663-4b57-b5e2-c903a81757ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:37.457666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23dd9d-9663-4b57-b5e2-c903a81757ab admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:37.457666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23dd9d-9663-4b57-b5e2-c903a81757ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:37.457666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23dd9d-9663-4b57-b5e2-c903a81757ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:37.458267 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23dd9d-9663-4b57-b5e2-c903a81757ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:37.459887 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23dd9d-9663-4b57-b5e2-c903a81757ab admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:37.470103 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1458/2889] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:37.474933 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a6be9-eb42-4b10-b1d5-f90a381993b0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:37.484888 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a6be9-eb42-4b10-b1d5-f90a381993b0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:37.490452 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeecc5a-de61-4e3e-a55b-acfd1ae485e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:37.491072 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeecc5a-de61-4e3e-a55b-acfd1ae485e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:37.491352 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeecc5a-de61-4e3e-a55b-acfd1ae485e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:37.491622 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeecc5a-de61-4e3e-a55b-acfd1ae485e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:37.497744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeecc5a-de61-4e3e-a55b-acfd1ae485e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:37.497744 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeecc5a-de61-4e3e-a55b-acfd1ae485e1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:37.504489 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1432/2890] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Sep 16 08:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 08:15:37.508688 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23dd9d-9663-4b57-b5e2-c903a81757ab admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:37.518480 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23dd9d-9663-4b57-b5e2-c903a81757ab admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:37.523610 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e1b33-550d-4145-b1eb-ebaa1b892347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:37.524260 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1b33-550d-4145-b1eb-ebaa1b892347 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:37.524537 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1b33-550d-4145-b1eb-ebaa1b892347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:37.524810 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1b33-550d-4145-b1eb-ebaa1b892347 admin admin] PATH_INFO: `/v3/projects/df0cf13aa30c4aa59e07e7b53018a1c0/users/1513ff3ac26f407b9659978ddd9a17de/roles/991609c9013b45b3907788ef9b42d051` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:37.540316 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e1b33-550d-4145-b1eb-ebaa1b892347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0cf13aa30c4aa59e07e7b53018a1c0, user_id=1513ff3ac26f407b9659978ddd9a17de, role_id=991609c9013b45b3907788ef9b42d051)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:37.543124 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e1b33-550d-4145-b1eb-ebaa1b892347 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:37.602941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1459/2891] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Sep 16 08:15:37 2022] PUT /identity/v3/projects/df0cf13aa30c4aa59e07e7b53018a1c0/users/1513ff3ac26f407b9659978ddd9a17de/roles/991609c9013b45b3907788ef9b42d051 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:37.614148 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b058c-89e9-4f92-9414-87ccc5e06cdd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:37.614515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b058c-89e9-4f92-9414-87ccc5e06cdd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:37.614832 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b058c-89e9-4f92-9414-87ccc5e06cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:37.646126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c66b058c-89e9-4f92-9414-87ccc5e06cdd None None] MFA Rules not processed for user `1513ff3ac26f407b9659978ddd9a17de`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:37.647421 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c66b058c-89e9-4f92-9414-87ccc5e06cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:37.712992 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1433/2892] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:37 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:37.724976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e1b33-550d-4145-b1eb-ebaa1b892347 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:37.727371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-977e1b33-550d-4145-b1eb-ebaa1b892347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:37.792825 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e1b33-550d-4145-b1eb-ebaa1b892347 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:37.796332 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe26ce-dcdf-47c0-9301-f3138520b295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb514134abe54e5ca8125e572a2ff928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:37.796937 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe26ce-dcdf-47c0-9301-f3138520b295 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:37.797213 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe26ce-dcdf-47c0-9301-f3138520b295 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:37.797484 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe26ce-dcdf-47c0-9301-f3138520b295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:37.805763 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe26ce-dcdf-47c0-9301-f3138520b295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:37.807241 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe26ce-dcdf-47c0-9301-f3138520b295 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:37.816131 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1460/2893] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:37 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:46.873229 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b058c-89e9-4f92-9414-87ccc5e06cdd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:46.934620 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b058c-89e9-4f92-9414-87ccc5e06cdd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:46.939097 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:46.939714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:46.939993 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:46.940264 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] PATH_INFO: `/v3/users/1513ff3ac26f407b9659978ddd9a17de` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:46.941373 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1513ff3ac26f407b9659978ddd9a17de)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:46.943379 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:47.117186 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1513ff3ac26f407b9659978ddd9a17de'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:47.128825 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1513ff3ac26f407b9659978ddd9a17de'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:47.136188 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1513ff3ac26f407b9659978ddd9a17de'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:47.137875 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:47.235518 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1513ff3ac26f407b9659978ddd9a17de'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:47.236126 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1513ff3ac26f407b9659978ddd9a17de'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:47.243254 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1434/2894] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:15:46 2022] DELETE /identity/v3/users/1513ff3ac26f407b9659978ddd9a17de => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:47.247771 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7b96a-a8dc-4468-ab91-728ace6c0fff None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:47.248263 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7b96a-a8dc-4468-ab91-728ace6c0fff None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:47.248371 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7b96a-a8dc-4468-ab91-728ace6c0fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:47.261638 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:47.267715 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cba7b96a-a8dc-4468-ab91-728ace6c0fff None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:47.268925 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cba7b96a-a8dc-4468-ab91-728ace6c0fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:47.326549 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1461/2895] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:47.335719 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:47.338272 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:47.399029 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3827d8-5d58-4915-a856-44ba0b7e88d2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:47.402989 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b23abd-8aa8-4a3a-a636-9a17335bd94b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:47.403600 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b23abd-8aa8-4a3a-a636-9a17335bd94b service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:47.403883 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b23abd-8aa8-4a3a-a636-9a17335bd94b service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:47.404157 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b23abd-8aa8-4a3a-a636-9a17335bd94b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:47.414132 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b23abd-8aa8-4a3a-a636-9a17335bd94b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:47.418308 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b23abd-8aa8-4a3a-a636-9a17335bd94b service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:47.425479 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1435/2896] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:47.894237 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7b96a-a8dc-4468-ab91-728ace6c0fff None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:47.895702 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cba7b96a-a8dc-4468-ab91-728ace6c0fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:47.908240 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7b96a-a8dc-4468-ab91-728ace6c0fff None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:47.913514 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:47.914179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:47.914462 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:47.914912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] PATH_INFO: `/v3/projects/df0cf13aa30c4aa59e07e7b53018a1c0` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:47.916055 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0cf13aa30c4aa59e07e7b53018a1c0)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:47.917983 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:48.001324 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0cf13aa30c4aa59e07e7b53018a1c0', 'request_id': 'req-6c78edf1-f27e-4025-9c28-67a58da8d5da'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:48.008086 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0cf13aa30c4aa59e07e7b53018a1c0', 'request_id': 'req-6c78edf1-f27e-4025-9c28-67a58da8d5da'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:48.013864 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0cf13aa30c4aa59e07e7b53018a1c0', 'request_id': 'req-6c78edf1-f27e-4025-9c28-67a58da8d5da'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:48.031918 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b23abd-8aa8-4a3a-a636-9a17335bd94b service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:48.033404 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-22b23abd-8aa8-4a3a-a636-9a17335bd94b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:48.045501 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b23abd-8aa8-4a3a-a636-9a17335bd94b service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:48.088143 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] The token cache is being invalidate because project df0cf13aa30c4aa59e07e7b53018a1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:15:48.088544 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:48.089209 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:15:48.090720 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1462/2897] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:15:47 2022] DELETE /identity/v3/projects/df0cf13aa30c4aa59e07e7b53018a1c0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:15:48.093313 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c53185-eead-49e2-b62e-02c8ad1d9656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:48.096396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c53185-eead-49e2-b62e-02c8ad1d9656 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:48.096396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c53185-eead-49e2-b62e-02c8ad1d9656 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:48.096396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c53185-eead-49e2-b62e-02c8ad1d9656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:48.096396 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a0c53185-eead-49e2-b62e-02c8ad1d9656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:48.107367 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c53185-eead-49e2-b62e-02c8ad1d9656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:48.108838 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c53185-eead-49e2-b62e-02c8ad1d9656 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:48.118409 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1436/2898] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 08:15:55.091220 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:55.093424 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:55.180745 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78edf1-f27e-4025-9c28-67a58da8d5da admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:55.186741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857545ec-ef18-4b13-8bfc-7fcba164c485 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:55.187409 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857545ec-ef18-4b13-8bfc-7fcba164c485 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:55.187696 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857545ec-ef18-4b13-8bfc-7fcba164c485 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:55.187970 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857545ec-ef18-4b13-8bfc-7fcba164c485 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:55.189755 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-857545ec-ef18-4b13-8bfc-7fcba164c485 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:55.204628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857545ec-ef18-4b13-8bfc-7fcba164c485 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:55.206251 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857545ec-ef18-4b13-8bfc-7fcba164c485 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:55.227410 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1463/2899] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:15:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:15:57.558195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340df7b4-e450-4f74-8c5d-2ac5ed63174b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:57.558195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340df7b4-e450-4f74-8c5d-2ac5ed63174b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:57.558195 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340df7b4-e450-4f74-8c5d-2ac5ed63174b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:57.606326 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-340df7b4-e450-4f74-8c5d-2ac5ed63174b None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:57.607572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-340df7b4-e450-4f74-8c5d-2ac5ed63174b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:57.672868 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1437/2900] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:15:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:57.777372 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857545ec-ef18-4b13-8bfc-7fcba164c485 service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:57.778815 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-857545ec-ef18-4b13-8bfc-7fcba164c485 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:57.797166 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:57.870142 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857545ec-ef18-4b13-8bfc-7fcba164c485 service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:57.875094 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd01bc-aabc-4578-b3f8-7161c2776a31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:57.875695 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd01bc-aabc-4578-b3f8-7161c2776a31 service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:57.875976 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd01bc-aabc-4578-b3f8-7161c2776a31 service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:57.876250 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd01bc-aabc-4578-b3f8-7161c2776a31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:57.884724 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd01bc-aabc-4578-b3f8-7161c2776a31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:57.886287 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd01bc-aabc-4578-b3f8-7161c2776a31 service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:57.894740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1464/2901] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:15:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:15:58.695817 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439504ba-5aa0-40fc-99e9-6c2d61525061 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:58.696295 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439504ba-5aa0-40fc-99e9-6c2d61525061 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:58.696412 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439504ba-5aa0-40fc-99e9-6c2d61525061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:58.715816 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-439504ba-5aa0-40fc-99e9-6c2d61525061 None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:58.717027 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-439504ba-5aa0-40fc-99e9-6c2d61525061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:58.724991 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1438/2902] 10.222.0.59 () {58 vars in 967 bytes} [Fri Sep 16 08:15:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:58.737574 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd01bc-aabc-4578-b3f8-7161c2776a31 service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:58.751758 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd01bc-aabc-4578-b3f8-7161c2776a31 service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:58.757068 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823611c6-827f-417e-ac70-4216df43b819 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:58.757740 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823611c6-827f-417e-ac70-4216df43b819 service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:58.758210 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823611c6-827f-417e-ac70-4216df43b819 service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:58.758487 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823611c6-827f-417e-ac70-4216df43b819 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:58.767437 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823611c6-827f-417e-ac70-4216df43b819 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:58.768968 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823611c6-827f-417e-ac70-4216df43b819 service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:58.778929 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1465/2903] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:15:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:15:59.280390 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa7801-9bac-42e9-9554-865f6fdb1055 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:59.280694 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa7801-9bac-42e9-9554-865f6fdb1055 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:59.281004 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa7801-9bac-42e9-9554-865f6fdb1055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:59.288593 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:15:59.300739 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6dfa7801-9bac-42e9-9554-865f6fdb1055 None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:59.302351 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6dfa7801-9bac-42e9-9554-865f6fdb1055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:59.309853 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1439/2904] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Sep 16 08:15:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:59.326564 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823611c6-827f-417e-ac70-4216df43b819 service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:59.332897 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823611c6-827f-417e-ac70-4216df43b819 service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:59.338058 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2921b-914b-443e-bd4b-28bd266ee064 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:59.338649 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2921b-914b-443e-bd4b-28bd266ee064 service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:59.338957 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2921b-914b-443e-bd4b-28bd266ee064 service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:59.339232 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2921b-914b-443e-bd4b-28bd266ee064 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:59.347312 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2921b-914b-443e-bd4b-28bd266ee064 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:59.348873 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2921b-914b-443e-bd4b-28bd266ee064 service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:59.356788 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1466/2905] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:15:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:15:59.906467 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fac5d-adce-413a-868c-6df054419790 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:59.906914 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fac5d-adce-413a-868c-6df054419790 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:59.907181 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fac5d-adce-413a-868c-6df054419790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:59.928698 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dd5fac5d-adce-413a-868c-6df054419790 None None] MFA Rules not processed for user `92d4614a1449473a9b949d4192cd2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:15:59.929925 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dd5fac5d-adce-413a-868c-6df054419790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:15:59.937392 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1440/2906] 10.222.0.59 () {58 vars in 967 bytes} [Fri Sep 16 08:15:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:15:59.949350 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2921b-914b-443e-bd4b-28bd266ee064 service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:15:59.960434 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2921b-914b-443e-bd4b-28bd266ee064 service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:15:59.965420 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a253b84-e054-4ca6-a257-f5961c1af16e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ae39fdd6ff4841843c9826dd88229a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:15:59.966051 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a253b84-e054-4ca6-a257-f5961c1af16e service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:15:59.966355 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a253b84-e054-4ca6-a257-f5961c1af16e service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:15:59.966629 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a253b84-e054-4ca6-a257-f5961c1af16e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:15:59.975538 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a253b84-e054-4ca6-a257-f5961c1af16e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:15:59.977140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a253b84-e054-4ca6-a257-f5961c1af16e service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:15:59.985177 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1467/2907] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Sep 16 08:15:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 08:16:02.639036 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fac5d-adce-413a-868c-6df054419790 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:16:02.644842 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dd5fac5d-adce-413a-868c-6df054419790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:16:02.714773 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fac5d-adce-413a-868c-6df054419790 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:16:02.720280 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:16:02.720855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:16:02.721225 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:16:02.721502 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] PATH_INFO: `/v3/users/012986457f7f42b295a3bca580bb7303` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:16:02.722775 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=012986457f7f42b295a3bca580bb7303)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:16:02.725249 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:16:03.077369 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '012986457f7f42b295a3bca580bb7303'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:03.089558 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '012986457f7f42b295a3bca580bb7303'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:03.096360 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '012986457f7f42b295a3bca580bb7303'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:03.184097 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '012986457f7f42b295a3bca580bb7303'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:03.187572 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '012986457f7f42b295a3bca580bb7303'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:03.191579 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1441/2908] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:16:02 2022] DELETE /identity/v3/users/012986457f7f42b295a3bca580bb7303 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:16:03.195866 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5a7ae-e656-4c0d-a3c2-4812b5c911d5 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:16:03.196347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5a7ae-e656-4c0d-a3c2-4812b5c911d5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:16:03.196476 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5a7ae-e656-4c0d-a3c2-4812b5c911d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:16:03.215837 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0ec5a7ae-e656-4c0d-a3c2-4812b5c911d5 None None] MFA Rules not processed for user `4763f3bdfee74904a41057be46f076ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 08:16:03.218714 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0ec5a7ae-e656-4c0d-a3c2-4812b5c911d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:16:03.278075 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1468/2909] 10.222.0.59 () {58 vars in 958 bytes} [Fri Sep 16 08:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 08:16:03.288674 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:16:03.290204 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:16:03.350381 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89843f9a-b7f4-4e21-9ed3-cbf4308e8bda admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:16:03.353938 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a465-fd2f-4b13-9dba-b10afaafd6c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:16:03.354546 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a465-fd2f-4b13-9dba-b10afaafd6c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:16:03.354898 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a465-fd2f-4b13-9dba-b10afaafd6c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:16:03.355176 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a465-fd2f-4b13-9dba-b10afaafd6c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:16:03.364140 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a465-fd2f-4b13-9dba-b10afaafd6c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:16:03.365561 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a465-fd2f-4b13-9dba-b10afaafd6c7 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:16:03.375573 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1442/2910] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:16:03.628488 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5a7ae-e656-4c0d-a3c2-4812b5c911d5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:16:03.631367 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0ec5a7ae-e656-4c0d-a3c2-4812b5c911d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:16:03.656016 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5a7ae-e656-4c0d-a3c2-4812b5c911d5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:16:03.657286 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:16:03.657926 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:16:03.658203 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:16:03.658478 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] PATH_INFO: `/v3/projects/189501ce59cd40bcbdf98964e256838f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:16:03.659627 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=189501ce59cd40bcbdf98964e256838f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:16:03.662522 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:16:03.780471 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:16:03.786789 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189501ce59cd40bcbdf98964e256838f', 'request_id': 'req-78def4b2-2b92-46ae-add6-30149b7f8996'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:03.793360 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189501ce59cd40bcbdf98964e256838f', 'request_id': 'req-78def4b2-2b92-46ae-add6-30149b7f8996'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:03.799043 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189501ce59cd40bcbdf98964e256838f', 'request_id': 'req-78def4b2-2b92-46ae-add6-30149b7f8996'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:03.881587 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] The token cache is being invalidate because project 189501ce59cd40bcbdf98964e256838f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:16:03.881883 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:03.882533 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:03.884140 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1469/2911] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:16:03 2022] DELETE /identity/v3/projects/189501ce59cd40bcbdf98964e256838f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:16:08.317432 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a465-fd2f-4b13-9dba-b10afaafd6c7 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:16:08.319503 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4154a465-fd2f-4b13-9dba-b10afaafd6c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:16:08.331083 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a465-fd2f-4b13-9dba-b10afaafd6c7 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:16:08.338861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:16:08.378752 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42ffb6-41a5-43c7-8ec9-f9c63a3a0868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:16:08.379348 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42ffb6-41a5-43c7-8ec9-f9c63a3a0868 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:16:08.379687 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42ffb6-41a5-43c7-8ec9-f9c63a3a0868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:16:08.379967 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42ffb6-41a5-43c7-8ec9-f9c63a3a0868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:16:08.382282 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3a42ffb6-41a5-43c7-8ec9-f9c63a3a0868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:16:08.444969 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42ffb6-41a5-43c7-8ec9-f9c63a3a0868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:16:08.446400 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42ffb6-41a5-43c7-8ec9-f9c63a3a0868 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:16:08.455946 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1443/2912] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:16:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:16:13.887119 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=9,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:16:15.363497 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:16:15.366004 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:16:15.385233 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78def4b2-2b92-46ae-add6-30149b7f8996 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:16:15.390381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:16:15.391118 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:16:15.391443 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:16:15.391680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] PATH_INFO: `/v3/users/afbb1d9c9bb34fbd8bfc78710c86d1f5` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:16:15.393303 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afbb1d9c9bb34fbd8bfc78710c86d1f5)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:16:15.395381 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:16:15.649528 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afbb1d9c9bb34fbd8bfc78710c86d1f5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:15.661754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afbb1d9c9bb34fbd8bfc78710c86d1f5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:15.668590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afbb1d9c9bb34fbd8bfc78710c86d1f5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:15.755725 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afbb1d9c9bb34fbd8bfc78710c86d1f5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:15.756347 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afbb1d9c9bb34fbd8bfc78710c86d1f5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:15.762781 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1470/2913] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:16:15 2022] DELETE /identity/v3/users/afbb1d9c9bb34fbd8bfc78710c86d1f5 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:16:16.009488 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42ffb6-41a5-43c7-8ec9-f9c63a3a0868 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:16:16.019601 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42ffb6-41a5-43c7-8ec9-f9c63a3a0868 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:16:16.078245 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:16:16.078901 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:16:16.079185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:16:16.079459 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] PATH_INFO: `/v3/projects/794ad972bfa844538f6f71d62e7bd163` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:16:16.080563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=794ad972bfa844538f6f71d62e7bd163)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:16:16.082461 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:16:16.212000 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '794ad972bfa844538f6f71d62e7bd163', 'request_id': 'req-ad15a4ee-09b4-431d-b640-6dcdad50d890'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:16.218394 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '794ad972bfa844538f6f71d62e7bd163', 'request_id': 'req-ad15a4ee-09b4-431d-b640-6dcdad50d890'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:16.224010 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '794ad972bfa844538f6f71d62e7bd163', 'request_id': 'req-ad15a4ee-09b4-431d-b640-6dcdad50d890'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:16.226153 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:16:16.297101 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] The token cache is being invalidate because project 794ad972bfa844538f6f71d62e7bd163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:16:16.297323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:16.297936 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:16.299414 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1444/2914] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:16:16 2022] DELETE /identity/v3/projects/794ad972bfa844538f6f71d62e7bd163 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:16:16.304341 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:16:16.305756 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:16:16.319167 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7e3c0-484d-4d7e-8866-73fb7715bc68 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:16:16.374891 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:16:16.375796 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:16:16.375796 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:16:16.376067 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] PATH_INFO: `/v3/users/6b1a524bce5847b3b08d0b841ced42f7` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:16:16.377179 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b1a524bce5847b3b08d0b841ced42f7)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:16:16.379121 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:16:16.523542 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b1a524bce5847b3b08d0b841ced42f7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:16.535886 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b1a524bce5847b3b08d0b841ced42f7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:16.543390 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b1a524bce5847b3b08d0b841ced42f7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:16.545079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:16:16.578105 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b1a524bce5847b3b08d0b841ced42f7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:16.578655 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b1a524bce5847b3b08d0b841ced42f7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:16.585090 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1471/2915] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:16:16 2022] DELETE /identity/v3/users/6b1a524bce5847b3b08d0b841ced42f7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:16:16.813981 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:16:16.816940 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:16:16.829714 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15a4ee-09b4-431d-b640-6dcdad50d890 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:16:16.885634 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:16:16.886266 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:16:16.886549 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:16:16.886861 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] PATH_INFO: `/v3/projects/8750ae6fd91c4232836b5360727849d5` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:16:16.888002 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8750ae6fd91c4232836b5360727849d5)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:16:16.889920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:16:17.019109 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8750ae6fd91c4232836b5360727849d5', 'request_id': 'req-fab3289b-9f39-4480-9d32-1b637843f207'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:17.026862 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8750ae6fd91c4232836b5360727849d5', 'request_id': 'req-fab3289b-9f39-4480-9d32-1b637843f207'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:17.032596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8750ae6fd91c4232836b5360727849d5', 'request_id': 'req-fab3289b-9f39-4480-9d32-1b637843f207'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:17.091397 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] The token cache is being invalidate because project 8750ae6fd91c4232836b5360727849d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:16:17.091619 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:17.092234 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:16:17.093708 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1445/2916] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:16:16 2022] DELETE /identity/v3/projects/8750ae6fd91c4232836b5360727849d5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:16:26.629851 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:16:27.088273 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=53 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:17:40.941321 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:17:40.942847 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:17:41.003407 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20ae8f-aa79-43f5-9005-7102b67ca331 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:17:41.050794 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3038fde-164b-4cc5-89f1-fc6786053a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:17:41.051400 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038fde-164b-4cc5-89f1-fc6786053a2f service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:17:41.051680 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038fde-164b-4cc5-89f1-fc6786053a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:17:41.051954 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038fde-164b-4cc5-89f1-fc6786053a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:17:41.053686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b3038fde-164b-4cc5-89f1-fc6786053a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:17:41.063354 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3038fde-164b-4cc5-89f1-fc6786053a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:17:41.064768 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3038fde-164b-4cc5-89f1-fc6786053a2f service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:17:41.072664 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1472/2917] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:17:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:17:41.144156 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:17:41.145692 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:17:41.158377 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3289b-9f39-4480-9d32-1b637843f207 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:17:41.161942 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d03ff-79ff-4be9-aa3f-2f670b64788f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:17:41.162870 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d03ff-79ff-4be9-aa3f-2f670b64788f service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:17:41.163161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d03ff-79ff-4be9-aa3f-2f670b64788f service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:17:41.163436 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d03ff-79ff-4be9-aa3f-2f670b64788f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:17:41.165118 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a55d03ff-79ff-4be9-aa3f-2f670b64788f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:17:41.236131 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d03ff-79ff-4be9-aa3f-2f670b64788f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:17:41.237481 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d03ff-79ff-4be9-aa3f-2f670b64788f service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:17:41.248323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1446/2918] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:17:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:17:41.311628 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3038fde-164b-4cc5-89f1-fc6786053a2f service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:17:41.313670 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b3038fde-164b-4cc5-89f1-fc6786053a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:17:41.382633 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3038fde-164b-4cc5-89f1-fc6786053a2f service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:17:41.387666 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf112e6-0b3f-4c88-87e2-77842df80b83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:17:41.388607 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf112e6-0b3f-4c88-87e2-77842df80b83 service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:17:41.388607 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf112e6-0b3f-4c88-87e2-77842df80b83 service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:17:41.388812 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf112e6-0b3f-4c88-87e2-77842df80b83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:17:41.390534 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6bf112e6-0b3f-4c88-87e2-77842df80b83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:17:41.402527 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf112e6-0b3f-4c88-87e2-77842df80b83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:17:41.404137 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf112e6-0b3f-4c88-87e2-77842df80b83 service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:17:41.413966 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1473/2919] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:17:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:17:42.134019 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d03ff-79ff-4be9-aa3f-2f670b64788f service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:17:42.136580 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a55d03ff-79ff-4be9-aa3f-2f670b64788f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:17:42.155596 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d03ff-79ff-4be9-aa3f-2f670b64788f service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:17:42.161817 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91758ee6-90e2-4077-b302-e3dd622583dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:17:42.166735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91758ee6-90e2-4077-b302-e3dd622583dc service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:17:42.166735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91758ee6-90e2-4077-b302-e3dd622583dc service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:17:42.166735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91758ee6-90e2-4077-b302-e3dd622583dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:17:42.166735 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-91758ee6-90e2-4077-b302-e3dd622583dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:17:42.177728 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91758ee6-90e2-4077-b302-e3dd622583dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:17:42.179345 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91758ee6-90e2-4077-b302-e3dd622583dc service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:17:42.190415 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1447/2920] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:17:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:17:51.409694 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=78 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:17:52.185584 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=40 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:19:13.177499 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf112e6-0b3f-4c88-87e2-77842df80b83 service placement] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:19:13.181590 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6bf112e6-0b3f-4c88-87e2-77842df80b83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:19:13.207097 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf112e6-0b3f-4c88-87e2-77842df80b83 service placement] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:19:13.212524 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa3fa5-f165-4947-8420-47e928fee0af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e32b5bf1a2524f1cbc432f38195e4c08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:19:13.213132 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa3fa5-f165-4947-8420-47e928fee0af service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:19:13.213414 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa3fa5-f165-4947-8420-47e928fee0af service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:19:13.213686 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa3fa5-f165-4947-8420-47e928fee0af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:19:13.215746 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8baa3fa5-f165-4947-8420-47e928fee0af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:19:13.228516 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa3fa5-f165-4947-8420-47e928fee0af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:19:13.230084 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa3fa5-f165-4947-8420-47e928fee0af service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:19:13.241043 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1474/2921] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:19:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:19:17.174631 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91758ee6-90e2-4077-b302-e3dd622583dc service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:19:17.177848 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-91758ee6-90e2-4077-b302-e3dd622583dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:19:17.190066 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91758ee6-90e2-4077-b302-e3dd622583dc service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:19:17.195312 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599073c9-aa33-4da9-9e63-d6d64efa95b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:19:17.195965 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599073c9-aa33-4da9-9e63-d6d64efa95b6 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:19:17.196242 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599073c9-aa33-4da9-9e63-d6d64efa95b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:19:17.196509 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599073c9-aa33-4da9-9e63-d6d64efa95b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:19:17.198161 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-599073c9-aa33-4da9-9e63-d6d64efa95b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:19:17.209822 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599073c9-aa33-4da9-9e63-d6d64efa95b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:19:17.211420 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599073c9-aa33-4da9-9e63-d6d64efa95b6 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:19:17.221782 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1448/2922] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:19:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:19:21.821300 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa3fa5-f165-4947-8420-47e928fee0af service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:19:21.833733 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa3fa5-f165-4947-8420-47e928fee0af service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:19:21.839053 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890d0ba-e0cc-4aa6-8677-bc3278e501a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:19:21.839737 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890d0ba-e0cc-4aa6-8677-bc3278e501a6 service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:19:21.840079 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890d0ba-e0cc-4aa6-8677-bc3278e501a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:19:21.840420 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890d0ba-e0cc-4aa6-8677-bc3278e501a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:19:21.842271 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2890d0ba-e0cc-4aa6-8677-bc3278e501a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:19:21.855398 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890d0ba-e0cc-4aa6-8677-bc3278e501a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:19:21.857566 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890d0ba-e0cc-4aa6-8677-bc3278e501a6 service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:19:21.887060 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1475/2923] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:19:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:19:23.611284 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599073c9-aa33-4da9-9e63-d6d64efa95b6 service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:19:23.621074 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599073c9-aa33-4da9-9e63-d6d64efa95b6 service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:19:23.625636 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398f821-bb68-4122-8c27-007d5d6f2292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c4e53ac9a438e9701a70f3de3f32e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:19:23.626275 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398f821-bb68-4122-8c27-007d5d6f2292 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:19:23.626563 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398f821-bb68-4122-8c27-007d5d6f2292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:19:23.626855 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398f821-bb68-4122-8c27-007d5d6f2292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:19:23.628700 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6398f821-bb68-4122-8c27-007d5d6f2292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:19:23.694877 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398f821-bb68-4122-8c27-007d5d6f2292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:19:23.696235 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398f821-bb68-4122-8c27-007d5d6f2292 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:19:23.706971 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1449/2924] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:19:25.719273 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890d0ba-e0cc-4aa6-8677-bc3278e501a6 service placement] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:19:25.720946 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2890d0ba-e0cc-4aa6-8677-bc3278e501a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:19:25.736003 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890d0ba-e0cc-4aa6-8677-bc3278e501a6 service placement] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:19:25.740637 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:19:25.741606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:19:25.741606 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:19:25.741818 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] PATH_INFO: `/v3/users/032d96be1a1146069299d83919adf7ce` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:19:25.742941 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=032d96be1a1146069299d83919adf7ce)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:19:25.744836 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:19:25.925741 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032d96be1a1146069299d83919adf7ce'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:25.938109 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032d96be1a1146069299d83919adf7ce'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:25.945031 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032d96be1a1146069299d83919adf7ce'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.003886 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032d96be1a1146069299d83919adf7ce'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.004449 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032d96be1a1146069299d83919adf7ce'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.010584 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1476/2925] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:19:25 2022] DELETE /identity/v3/users/032d96be1a1146069299d83919adf7ce => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:19:26.222788 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398f821-bb68-4122-8c27-007d5d6f2292 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:19:26.233171 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398f821-bb68-4122-8c27-007d5d6f2292 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:19:26.289796 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:19:26.290331 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:19:26.290612 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:19:26.290911 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] PATH_INFO: `/v3/projects/715b5f5887c940aca9d81291d48fd426` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:19:26.292014 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] RBAC: Authorizing `identity:delete_project(project_id=715b5f5887c940aca9d81291d48fd426)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:19:26.293912 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:19:26.420279 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '715b5f5887c940aca9d81291d48fd426', 'request_id': 'req-80af9b4a-c69a-4e97-8c50-c061be3e939c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.426894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '715b5f5887c940aca9d81291d48fd426', 'request_id': 'req-80af9b4a-c69a-4e97-8c50-c061be3e939c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.432427 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '715b5f5887c940aca9d81291d48fd426', 'request_id': 'req-80af9b4a-c69a-4e97-8c50-c061be3e939c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.494323 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] The token cache is being invalidate because project 715b5f5887c940aca9d81291d48fd426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:19:26.494894 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.495515 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.496982 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1450/2926] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:19:26 2022] DELETE /identity/v3/projects/715b5f5887c940aca9d81291d48fd426 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:19:26.501672 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:19:26.503132 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:19:26.516185 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4ea7-a665-4107-b670-ccfb7a399416 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:19:26.572646 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:19:26.573199 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:19:26.573481 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:19:26.573757 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] PATH_INFO: `/v3/users/9b28a738458a4be7bbbf32f111d6e201` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:19:26.574905 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b28a738458a4be7bbbf32f111d6e201)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:19:26.576844 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:19:26.790546 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b28a738458a4be7bbbf32f111d6e201'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.791754 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:19:26.803565 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b28a738458a4be7bbbf32f111d6e201'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.810218 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b28a738458a4be7bbbf32f111d6e201'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.884062 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b28a738458a4be7bbbf32f111d6e201'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.884632 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b28a738458a4be7bbbf32f111d6e201'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:26.890645 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1477/2927] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Sep 16 08:19:26 2022] DELETE /identity/v3/users/9b28a738458a4be7bbbf32f111d6e201 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:19:27.102753 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:19:27.104185 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:19:27.119701 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af9b4a-c69a-4e97-8c50-c061be3e939c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:19:27.128238 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:19:27.176837 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4763f3bdfee74904a41057be46f076ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1dfa255cacf3473b9f6619fec53b8cf1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:19:27.177465 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:19:27.177740 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:19:27.178011 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] PATH_INFO: `/v3/projects/05bcf5ce4a1a47778e18c1ddbc7ecf59` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:19:27.179160 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] RBAC: Authorizing `identity:delete_project(project_id=05bcf5ce4a1a47778e18c1ddbc7ecf59)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:19:27.181483 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:19:27.276951 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05bcf5ce4a1a47778e18c1ddbc7ecf59', 'request_id': 'req-420b88e1-92fa-4e46-bb5a-58af1087674d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:27.283920 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05bcf5ce4a1a47778e18c1ddbc7ecf59', 'request_id': 'req-420b88e1-92fa-4e46-bb5a-58af1087674d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:27.289364 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05bcf5ce4a1a47778e18c1ddbc7ecf59', 'request_id': 'req-420b88e1-92fa-4e46-bb5a-58af1087674d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:27.349939 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] The token cache is being invalidate because project 05bcf5ce4a1a47778e18c1ddbc7ecf59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 08:19:27.350227 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:27.350873 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-420b88e1-92fa-4e46-bb5a-58af1087674d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 08:19:27.352380 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1451/2928] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Sep 16 08:19:27 2022] DELETE /identity/v3/projects/05bcf5ce4a1a47778e18c1ddbc7ecf59 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 08:19:37.106945 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=12,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:19:37.408440 dv-cinder-740146-20-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 08:24:34.552318 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 08:24:34.553762 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:24:34.692256 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091204f-2854-4b61-bf4c-5ee7fa90b228 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 08:24:34.752965 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7a2a9-2fc4-4edb-a623-a8f84d020ac4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2dca05dfe8b4af0a02ea93fbe29c556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7731b013b1c473fae443eb4dec76bc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 08:24:34.755912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7a2a9-2fc4-4edb-a623-a8f84d020ac4 service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 08:24:34.755912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7a2a9-2fc4-4edb-a623-a8f84d020ac4 service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 08:24:34.755912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7a2a9-2fc4-4edb-a623-a8f84d020ac4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 08:24:34.755912 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-07d7a2a9-2fc4-4edb-a623-a8f84d020ac4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 08:24:34.768084 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7a2a9-2fc4-4edb-a623-a8f84d020ac4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 08:24:34.769692 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7a2a9-2fc4-4edb-a623-a8f84d020ac4 service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 08:24:34.779674 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1478/2929] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Sep 16 08:24:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 08:24:44.775345 dv-cinder-740146-20-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}